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ABSTRACT 

/ The National Science Foundation initiated tha Science 

Development Program in 1965 in order to enable selected universities 
to imj)rove the quality of their resources for science education and 
research. The specific purpose was to develop the competence of the 
funded institutions in preparing research scholars (as certified by 
their awarding of the Doctor of Philosophy degree) and in conducting^ 
research projects (as evidenced by their receipt of research grants). 
The Science Development program awarded over $230 million to 31 
universities. The leading research universities in the country (the 
so-called "top 20") were excluded from participation*, as were 
institutions deemed too weak to advance r'^pidly. The institutions 
selected were those that seemed to have potential for developing 
their science capability markedly in a short period of time. This 
eva luative study assesses the overall effects of tne Science 
Development program, using nonfunded American doctorate-producing 
universities as controls. The major indicators were: faculty size, 
faculty mobility, scholarly productivity, graduate school enrollment 
and quality, Ph.D. production, and postdoctcrate eiiployment. 
(Author/MLH) 
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Highlights 



Quantitative analyses of longitudinal data, supplemented by site visits, 
were used to evaluate the impact of the NSF Science Development 
program. Setting as its twin goals a dramatic upgrading in the science 
capabilities of second-tier universities and a broader geographical dis- 
tribution of scientific talent throughout the nation, this funding program 
awarded over $230 million to selected universities during the 1960s. 

Several technical decisions were made at the outset to help isolate the 
unique effects of Science Development. First, wherever possible, the data, 
gathered for this study covered the 15 years from 1958 through 19^. 
Second, all (nonfunded) doctorate-producing universities in the country 
were used as controls. Third, the three science fields that received the 
largest share of Science Development funds — physics, chemistry, and 
mathematics— were analyzed, as was the nonfunded control field of 
history. Finally, to define * 'quality science education" in American 
graduate schools, multiple indicators (i.e., multiple criteria) were used. 

The m£Uor findings: 

{.Faculty Size National Science Foundation funds helped depart- 
ments in ail three science fields to enlarge their faculties. In physics and 
chemistry, this increase in faculty size was limited to the public sector 
(Chapter 5). 

2. Faculty Mobility An analysis of senior faculty mobility in the field of 
physics showed that the funded institutions did not develop by recruiting 
extensively from the leading physics departments. 
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3. Scholarly Productivity Science Development funding had a posi- 
tive effect on scholarly productivity as measured by rates of publication in 
key journals. That is, the funded departments registered an increase in the 
number of articles published by their faculty members in journals that 
have the most impact. This increase, however, was largely a function of 
the growth in faculty size; the effects on the publication rate of the 
individual faculty members were minimal (Chapter 6). 

4. Graduate Student Enrollment and Quality Receipt of a grant was 
not closely related to increases jn graduate student enrollments. Funded 
departments, however, were able to attract higher quality graduate 
students (as measured by an improvement in the scores of first-year 
graduate students on the Graduate Record Examination), though there 
was no change in the quality of graduate students if one judges by the 
selectivity of their baccalaureate institutions (Chapter 7). 

5. Ph.D. Production Although Science Development funds increased 
the production of Ph.D.s in physics^iji mathematics that effect was felt 
only in the public sector, not the private. In chemistry, no impact at all 
was apparent (Chapter 8). 

6. Postdoctorate Employment Ph.D.s from funded institutions dif- 
fered very little from Ph.D.s from nonfunded institutions with respect to 
the attractiveness of the jobs they took, whether in academe or outside of 
academe, after receiving the degree. New Ph.D.s in mathematics from 
private Science Development institutions were somewhat more successful 
than those from (private) control institutions in getting positions at high- 
quality universities (Chapter 8). 
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Foreword 



This technical report on the Science Development program, by David 
Drew, is an experiment in institutional evaluation. Through the program, 
the National Science Foundation hoped to enable selected universities to 
improve substantially the quality of their resources for science education 
• and research. The purpose was to develop the competence of the funded 
institutions in preparing research scholars (as certified by their awarding 
of the Doctor of Philosophy degree) and in conducting research projects 
(as evidenced by their receipt of research grants). 

The institutions eligible to apply for the Science Development program 
when it was initiated in 1965 were those considered to be in the * 'second 
tier" with respect to quality and reputation in science education. The 
leading research universities in the counuy (the so-called ''top 20") 
were excluded from participation, as were institutions deemed to be too 
weak to advance rapidly. The institutions selected, then, were those that 
seemed to have potential for developing their science capability maikedly 
in a rather short period of time. Presumably with the assistance of a 
Science Development grant, these institutions would advance toward 
excellence; thus, the National Science Foundation hoped to broaden the 
base of outstanding science competence available to the federal govern- 
ment for its national purposes and to the citizens of the United States. 
Science Development had as its goal institution bOilding. 

The program was by no means unique. Late in the 1940s, both the 
Carnegie Corporation of New York and The Rockefeller Foundation had 
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made grants designed to help certain universities to expand their pro- 
grams, to achieve a higher level of quality, and to set standards of 
excellence that would guide other universities. In the 1950s, the Ford 
Foundation, with its Challenge Grants, supported a similar endeavor. 
What was different in I96S was that this new enterprise in institution 
building was undertaken by an agency of the federal government. 

Dr. Drew was given the problem of how to evaluate the accomplish- 
ments of the Science Development program. What impact did the grants 
have upon the recipient institutions? Did they make possible the achieve- 
ment of a new level of excellence in science education and research? Did 
the recipients in fact develop into centers of excellence by reason of these 
grants? These were the questions to which Dr. Drew sought answers. 

It should be observed here that it was the National Science Foundation 
itself that asked for an evaluation of Science Development. The program 
having been completed, NSF wanted to know what it had accomplished. 
How had the nation benefited from the selection of 31 institutions to 
receive University Science! Development grants? The National Science 
Foundation is to be commended for its spirit of self-inquiry and its 
initiative in requesting an examination of the consequences of its action. 

The difficulty in this circumstance is that evaluating institutional 
accomplishment is no pimple matter. More is involved than determining 
how the infusion of NSF funds affected the quality of student output or the 
** value added" to student input, or the quality of particular graduate 
programs. Essentially, the problem was to determine what happened to a 
university once the impact of Science Development grants began to be 
felt. Just how does one determine that changes in quality have occurred at 
a university and that these changes can be traced or attributed to the 
availability of additional financial resources? What kinds of objective 
evidence does one look for? 

In this study. Dr. Drew has had to develop techniques of evaluation as 
well as apply the e techniques to the problem at hand. I am sure that he 
would be the last person to claim that the standards he has used to 
determine the achievements in a particular university over a particular 
period of time are ultimate and infallible. Nevertheless, I myself believe 
that, working within the constraints of the available data. Dr. Drew has 
notably advanced the slate of the art of institutional evaluation. 

In the years ahead, we who labor in the field of higher education will be 
confronted as never before with demands of two kinds. First, we shall be 
asked to justify the belief that quality in science education and research, 
quality in all higher education and research, is important to the national 
welfare. And second, we shall be asked to prove that the amount of 
financial support available to a university is related to the quality of the 
achievements of that university. I pass over still another issue: If quality 
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is so important, and if financial support is the key to quality, then who in 
society should be expected to pay? But those of us who believe strongly in 
high standards or achievement in hi^ her education shall be expected to 
demonstrate the social benefits that derive from excellence and to 
articulate the means to excellence. 

No longer can we accept the interrelation between financial support 
and academic excellence as an article of faith. We must prove beyond a 
reasonable doubt that this faith is grounded in reason and fact, not in 
ancient dogma. 

I would like to think that, faced with these concerns in the future, we 
can look back to this study by Dr. Drew and draw from it encouragement: 
proof that the interrelation is there if we know how to look for it; evidence 
of quality in an institution, and of its connection with financial support, 
can be identified. ^ 

Institutional evaluation is here to stay. Now we must strive to make it 
ever more reliable. This study is a start in the right direction. 

John D. Millett 
Chairman of the Panel 
December 1974 . Science Development Evaluation Study 
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At the outset of this endeavor, it was apparent to me that assessing the 
Science Development program offered a rare and stimulating opportunity 
to the evaluation researcher. This National Science Foundation program 
of the 1960s had been a revolutionary experiment in institutional financ- 
ing. Millions of dollars were poured into a limited number of carefully 
"selected universiies with the goaJ of dramatically improving their 
graduate science capabilities. The challenge of evaluating Science De- 
velopment struck me as particularly exciting in two ways, j 

First, it offered the chance to conduct research that might have some 
impact on future federal policy. However much I and my colleagues in the 
research community would like to believe otherwise, the fact of the 
matter is that most educational research is ignored by policymakers. This 
study had the potential to become one of the fexceptions. For one thing, 
the program had dispensed significantly more money than most planned 
interventions in higher education and had done so in a unique, d||matic 
way. Also, Science Development had only recently been terminated, and 
its memory was still fresh; many scientists and policymakers were asking 
questions about its effectiveness and about the lessons that might be 
learned from it. Moreover, the study had been requested by the evalua- 
tion unit of the National Science Foundation itself— an encouraging sign. 
The stjij^y was housed in the National Board on Graduate Education 
(NBGJ^^hich in turn is housed in the National Research Council. The 
sponsorship of the study by these groups,* each of which carries weight in 

^Financially, this research was supported in its entirety by the evaluation unit of the 
National Science Foundation under Contract C 310, Task Order 263. 
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the academic and scientific communities, meant that the results would 
reach a wide and interested audience. 

The second challenge was methodological. The field of assessment 
research is replete with what statisticians refer to as type I and type II 
errors: on the one hand, subjective case studies that infer dubious 
•^effects"; on the other, sophisticated empirical analyses that consistently 
fail to detect any effects whatsoever. One frequent source of the latter 
problem is that the funds given under the particular program being 
evaluated represent only a small drop in ^ very large bucket insofar as the 
recipients are concerned. But ScicpccDcvelopment was different. Large 
sums of money (approximately $6^ million per school) were given to only a 
few universities (31 in all) for a specific purpose. If the program worked, 
its effects should be discernible. Further, Science Development consti- 
tuted a natural experiment in that some universities received funds while 
other, roughly comparable, institutions did not. 

In short, this study presented the opponunity to conduct a thorough 
assessment and perhaps to contribute to the development of evaluation^ 
methodology. These objectives (combined with some biases of my own as 
a methodologist) led to certain technical decisions thai are discussed at 
length in this volume. For instance, longitudinal data were deemed 
essential for the quantitative analyses. In addition, the decision to use 
multiple criteria of quality made it possible to study differential effects 
and thus to avoid reducing the study to one generalj stamp of approval or 
rejection. At the same time, the quantitative analyses were supplemented 
with intensive case studies to explore the way these effects operated in 
the "real worid,** thereby obviating the danger of turning the study into a 
self-contamed mathematical exercise (as too many multivariate, quantita- 
tive studies are). Finally, recognizing the many valid criticisms^ of 
assessment studies where the control group provided an inappropriate 
comparison, we decided to use all m^or doctorate-producing universities 
in the nation as the controls. 

Conducting the study did, indeed, turn out to be an exciting and 
chdJlenging enterprise. As the reader will see, we found that Science 
Development had a number of effects — some good, some bad. It is our 
hope that some of the techniques developed here will prove useful to 
other researchers and that the results of this study ^will be a guide for 
future federal policy. With that goal in mind, the National Board on 
Graduate Education, with the assistance of the study's Advisory Panel, 
has issued a companion volume to this report in which the poh'cy implica- 
tions of the res'jarch results are spelled out.* 

'National Board on Graduate Education. Science Development, University Development, 
and the Federal Government (Washington. D.C.. National Academy of Sciences. 1975). 
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Science Development: 

An Evaluation Study 



Introduction . 



Thir report presents the findings of an extensive evaluation study of the 
National Science Foundation's Science Development program. The fed- 
eral government launched this massive funding program in 1%5, Intended 
to improve graduate science education in the United States, it was based 
on three innovative concepts. First a group of second-tier universities — 
those that ranked just below the **top 20" in science quality — should be 
given substantial infusions of funds to propel them into becoming **cen- 
ters of excellence." 

Second, support should be given to the institution to be targeted at 
science areas. This pattern contrasts with previous patterns of federal 
funding directed at individual researchers or projects. Such awards, it was 
argued, tended to create scientific prima donnas with little institutional 
loyalty^ and benefited the institution only indirectly, if at all. 

Th6 final concept was thai every region in the country should be served 
by an outstanding university, a notion strongly supported by Presidents 
Kennedy and Johnson. Previous national studies, notably the Seaborg 
report (1%0),* had commented on the intense concentration of federal 
support for sciehce in a few select universities, disproportionately clus- 
tered in the Northeast. 

Although the structure of the funding program changed several times, 
the major subprogram was University Science Development (USD), under 

^President's Science Advisory Commillee. Snenti/ic Progress, the Universities, and the 
Federal Government (Washington. fhe While .House. November 1960). 
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which 31 universities received over $177 million. These institutions had to 
give evidence of having an overall development scheme that included 
extensive plans for the sciences. Technically, matching funds were not 
required, but the institution was expected to make a significant financial 
\ contribution as part of the program. The National Science Foundatioji 
(NSF) excluded from consideration not only existing centers of excellence 
but also institutions that it felt lacked sufficient strength to develop 
rapidly into such centers. 

There were two other subprograms relating to graduate science educa- 
tion under which limited grants were given for less ambitious develop- 
ment plans: Department Science Development (DSD) and Special 
Science Development (SSD). Both are described in greater detail in 
Chapter 2. 

As noted above, the Science Development program differed from most 
federal support of university activities in that the money did not go 
directly to individual scientists or scholars and their projects, but to 
institutions. The program also differed from other more traditional forms 
of institutional support such as formula grants— notably, in the relatively 
lar^e size of the awards. Science Development funds were used for hiring 
new faculty, graduate student support, the development of interdiscipli- 
nary institutes, the construction of major new facilities, and so forth. 

Federal expenditures for science research have been commonplace 
since World War II and the spectacular technical success of the Manhat- 
tan project. Shortly after the war, the case for continued government 
support of basic science research was made by Vannevar Bush^ and 
others; the major science organization tp grow out of this federal concern 
was the National Science Foundation. Subsequently, in the late 1950s 
(with the voyage of Sputnik), science education became a national 
priority. That period spawned a wide^ array of measures in support of 
science education (e.g., the National Defense Education Act). 

With the passage of time, the government grew increasingly reluctant 
simply to underwrite projects with a biank check and, concomitantly, 
became more concerned with monitoring federally supported programs. 
Thus, for example, the landmark 1%5 Elementary and Secondary Educa- 
tion Act contained measures requiring that the projects it was launching 
be evaluated. 

Often, ''evaluation" means no more than a cursory field trip followed 
by armchair speculation resulting in a plea for more funds, thinly 
disguised as an accountability report. Against this background, the 
request by the NSF evaluation unit for a major empirical assessment of the 
effects of Science Development stands in sharp relief. 
'Vannevar Bush. Science. The Endless Frontier: A Report to the President (Washington, 
D.C.: U.S. Government Printing Office. July 1945). 
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In 1971 (the same year that saw the demise of the Science Development 
program), NSF asked the National Academy of Sciences to conduct this 
evaluative study under funding from the Foundation. The most appro- 
priate location for the study within the complex Academy structure was felt 
to be the National Board on Graduate Education. The Board, chaired by 
David Henry, was established by the Co|iference Board on Associated 
Research Councils to **provide a means for an unbiased, thorough 
analysis of graduate education today."^ Both Academy officials and 
National Board members agreed that it was vital to underscore at the 
outset the necessity for an objective, scientific assessment. Further, both 
groups agreed that the study woulil be relevant to their charters as long as 
the focus was not on a limited evaluation or audit of the funding progranri 
but rather on a more general study that would have wider policy 
implications. 

The basic goal, of course, was to examine the effectiveness of this form 
of institutional support: Did the funded institutions develop their science 
capabilities at a significantly greater rate in the past decade than did 
nonfiinded institutions (including the '*top 20")? What were the effects of 
the program's emphasis on geographical dispersion? What was the 
impact, if any, on nonfunded departments in institutions that received a 
grant? What happened in the funded universities when these massive 
government grants ended? What were the differential effects of funds 
given under the USD and DSD subprograms? 

To answer these questions, we had to tackle the thorny issue of 
* defining "quality" science education. What does this term mean as 
applied to graduate schools in the United States? It rapidly became clear 
that no single index would do; we needed to use multiple indicators of 
quality; i.e., multiple criteria. It is hoped tiiat this research will contribute 
toward'a definition of "quality graduate education" in terms that can be 
operationalized for empirical research. 

Several principles guided the development of the methodology for this 
evaluation study: 

1. The research combined two approaches: (a) multivariate analyses of 
quantitative data on institutions and (b) selected case studies, carried out 
by site visits. 

2. All too frequently, evaluation studies of complex projects try to 
rcouce their conclusions to one general stamp of approval or rejection. 
The goal of this study was both more complex and more difficult: to tiace 
the differential effects of funding patterns upon a number of outcomes 
represented by various indices of science excellence. The ultimate goal 
Hjraduale Education: Purposes, and Potential: A Report of the National Board on 
Graduate Education (Washington. D.C.: NBGE. November 1972), p. v. 
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was to develop policy recommendations that might guide future funding of 
graduate science education. 

3. Discerning the unique effects of Science Development required both 
longitudinal data and complex statistical techniques. Therefore, wherever 
possible, the information gathered for this study was based on the 15 
years from 1958 through 1972. Thus, not only did we span the entire 
lifetime of Science Development (1%5-1971) but also we collected data 
on trends under way before t%5. 

4. To allow for thorough and meaningful comparison of trends in the 
funded institutions with trends in other universities, we collected and 
analyzed data not just on funded institutions and a few select controls but 
on every major doctorate-producing institution in the nation. Thus, the 
sample consisted of all universities rated by Roose and Andersen in their 
1970 evaluation of graduate education in the United Stales.^ This proce- 
dure enabled us to compare the Science Development institutions with 
control institutions of roughly similar initial quality, as well as with those 
institutions excluded from the grant program either because they were 
already considered excellent or because their science capabilities fell well 
below those of the funded institutions. 

At this juncture it might be helpful to indicate how this technical report 
is organized. Chapter 2 contains a detailed history and description of the 
NSF Science Development program. Chapter 3 spells out the method? used 
in both the quantitative analyses and the site visits and indicates the 
reasons underlying several major technical decisions (e.g., sampling 
techniques and the heavy reliance on longitudinal data). , 

The site visits provided a rich resource for understanding many of the 
relationships uncovered in the quantitative analyses. In addition, some 
general observations about the impact of the program and about indi- 
cators of success and nonsuccess were made during those visits. Those 
observations are presented in Chapter 4. 

The next four chapters report the results of the quantitative analyses. A 
separate chapter is devoted to each of the parameters of science excel- 
lence. Chapter 5 describes the changes in faculty size resulting from the 
program; a special section examines tne validity of the charge leveled by 
some critics that Science Development funds did no more than permit 
recipient institutions to develop by stealing top talent from the leading 
institutions. In Chapter 6 the research productivity of Science Develop- 
ment faculty is explored through an analysis of publication rates in leading 
journals. (The procedure used to select the leading journals in each field is 
described in Appendix B.) Chapter 7 explores the effects of Science 

^Kenneth D. Roose and Charles J. Andersen. A Rating oj Graduate Programs (Washington, 
D.C.: American Council on Education. 1970). 



Development funding on the graduate student population in terms of 
enrollments, test scores, and baccalaureate origins. The program's impact 
on Ph.D. production is discussed in Chapter 8; because the overproduc- 
tion of Ph.D.s— and their subsequent underemployment— has become a 
hot topic in the last several years and because the'Science Development 
program was accused by some of contributing to the oversupply of 
Ph.D.s, a special analysis of the nature of postdoctorate employment was 
carried, out and is discussed here. The final chapter summarizes the 
methods and major findings of the study. 

There are three appendices. Appendix A lists the schools comprising 
each sample group for the study. In Appendix B the methods used to 
select the journals for the productivity anrfysis (Chapter 6) are presented. 
Finally, Appendix C reports the findings from a series of special 
supplementary analyses of the data. 

This technical report is the first of two documents to emerge from the 
study. The second is- a National Board on Graduate Education policy 
statement written by a task force comprising the members of the study's 
Advisory Panel, headed by John Millett. That statement summarizes the 
findings from the study, discusses some of its implications for graduate 
education in the United States, and enumerates policy guidelines that 
should prove valuable for future funding programs. 



The NSF Science 
Development Program 



Just as Research Applied to National Needs (RANN) has been the dominant 
new NSF program of the 1970s, so Science Development was the dominant 
new NSF program of the 1960s. In basic philosophy it represented a radical 
departure from previous federal funding for science. As indicated in 
Chapter 1, the program embodied three innovative concepts: 

1. Funding via institutional rather than project support; 

2. Funding of **second-tier" institutions (i.e., those not yet considered 
excellent) together with deliberate exclusion of those universities in the 
"top 20"; 

3. Strong emphasis on geographical dispersion of the funded institu- 
tions. 

To give a complete frame of reference in which to present the results 
of this evaluation study, the history and development of this NSF program 
will be traced. 

Perhaps the major impetus for a program of this type was Scientific 
Progress, the Universities, and the Federal Government, a statement by 
the President's Science Advisory Committee that appeared in November 
1960. A major conclusion of this report, known as the Seaborg report (after 
Glenn T. Seaborg, then Chancellor of the University of California, 
Berkeley, and chairman of the committee), follows: 



Equally with the importance of lusUumni what is already outstanding, we urge the i mportance 
for the country of an increase in the number o( universities in which ftnt-rate research and 
graduate teaching go forward together. The growth of science requires more places with 
supenorfacuities and outstanding groups of students. Existing strong institutions cannot fully 
meet the nation* s future needs. It is true that experience is casting doubt on some conservative 
notions about the optimum size of the university, and the universities which are already great 
are larger than they expected to be ten years ago. But there is a limit to such growth, and we 
must hope that where there were only a handful of generally first-rate academic centers of 
science a generation ago and may be as many as fifteen or twenty today, there Will be thirty or 
forty in another fifteen years. Timely and determined support to the rising centers will be 
repaid many times over in service to society.^ 

This observation was formalized in the Third of the general recommenda- 
tions: 

It is of equal importance to increase support for rising centers of science. Over the next fifteen 
years the Um'ted Sutes should seek to double the number of universities doing generally 
excellent work in basic research and graduate education.* 

This notion, first articulated while Eisenhower was still president, was 
subsequently endorsed and expanded by both the Kennedy and the 
Johnson administrations. The program took more definite form during 
Johnson's administration and was consistent with h's philosophy of 
geographic dilBfusion of funds. In fact a 1%5 Executive Order on that 
subject was closely tied to the disbursement of NSF funds for University 
Science Development. 

The Science Development program was formally announced in March 
1964; the first grants under the program were awarded during fiscal year 
1965. These awards, to eight universities (two of which. Case and Western 
Reserve, subsequently merged) were msoor institutional grants designed to 
upgrade a number of science departments in each of the recipient 
institutions. At that point, NSF was awarding funds only in the mode that 
subsequently came to be known as the University Science Development 
subprogram. . In an excellent article reviewing the history of Science 
Development, Howard E. Page, a former administrator of the program, 
discusses the philosophy underiying institutional support: 

Heretofore, nsf prt)grams had been concerned mainly with the development of individual 
scientisU. the expansion of scientific knowledge, and the building of research laboratories. 
Valuable as these programs were, their aim was to strengthen science per se; they did not 
necessarily aim at strengthening the institutions that furnish the home for science. To think 
institutionally meant to think more broadly than the foundation had done before. An 
institutional program must provide a means of integrating the various components of the 
customary kinds of support into an organic unit. Activities in natural science ahd engineering 

« President's Science Advisory Committee. Scientific Progress, the Umyerstties, and the 
Federal Government (Washington. D.C.: The White House. November IV60). pp. 14-15. 
'/6W..P.28. 
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must be viewed in relation to those in the humanities and social sciences; the ways by which an 
institution's educational and research policies are determined and administered must be 
understood; the present and potential sources of the institution's financial support iiiust be 
known; and the hopes and plans of the institution's its special cuUure, and its relation 'o its 
social and intellectual environment must be perceived. In other words, a foundation oHicial 
must put himself behind the desk of the college or university president, at the same time 
remembering that, as a guardian of federal tax funds, he is concerned about the general health 
of science rather than the advancement of one institution alone.' 

While Science Development was a significant innovation, its developers 
drew to a certain degree on the experience of the Ford Foundation in its 
Challenge Grant program. There were, however, differences between the 
two programs. First, the Ford grants were restricted to private i»>stitutions 
and were given only to schools the foundation invited. Second, Ford grants 
were given to both universities and four-year colleges in larger amounts 
than were provided by the NSF program; for example, the amount awarded 
to some universities went as high as $25 million, roughly three times as 
much as the largest Science Development grant. Finally, the Ford money 
was not restdfted to science fields. 

A federal precedent for institutional support of higher education is 
provided by the 1887 Hatch Act, administered by the Department of 
Agriculture. This Act established the experimental stations at land-grant 
colleges to advance the agricultural sciences and to disseminate informa- 
tion, particulariy among rural communities, about discoveries in the field. 



STRUCTURE OF SCIENCE DEVELOPMENT 

In the first years of the program, it became apparent that the pat*ern 
followed in awarding the early grants would not fit every institutional 
situation encountered by NSF. As a result, the program underwent several 
reorganizations that resulted in a series of subprograms. ' 

The primary form of institutional funding^wards of three-year grants 
amounting to approximately $4 million each (with two-year supplementary 
increments>-4)ecame known as the University Science Development 
(USD) subprogram. The USD approach essentially embodied the Science 
Development philosophy; under this subprogram, by far the greatest 
amount of money was awarded ($177 million to 31 institutions). 

A second subprogram, the Special Science Development (SSD) subpro- 
gram, first awarded grants in fiscal year 1966. The purpose of these special 
grants was to fund institutions that originally had applied for a USD grant but 

* Howard E. Page. "The Science Development Program.** in Science Policy and the 
University, ed. Harold Orlans (Washington, D.C.: Brookings Institulion. 1968), p. 103. 
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that the Foundation judged lacking in sufficient overall science strength to 
justify a total instHutional award. Thus, SSD grants were given to one or two 
' science departments instead of the five op., six that the institution might 
originally have proposedjn their USD application. Under this subprogram, 
a total of 1 1 grants were awarded in the total amount of $11,937,000. 

The third program, the Departmental Science Development (dsd) 
subprogram was announced in October 1966 and became operational in 
January 1%7. Ultimately, 73 grants averaging about $600,000 each (for a 
total of $41,932,210) were given to departments in 62 institutions. This 
subprogram served se verafsuccessive purposes. At first, it was intended in 
part to boostinstituti0nsih urban areas that otherwise lacked good science 
facilities. Although eventually the Foundation funded schools in all types 
of demographic settings, a sizable number of the DSI>grants did go to 
institutions in urban centers of 250,000 or* more. Perhaps the key 
characteristic of this subprogram was the funding of a single department, 
occasionally two, that showed the capacity to develop excellence in 
an institution that otherwise demonjjlrated mediocre science capabilities. 
Departmental Science Development awards began as proposals&r support 
of a single department, unlike the SSD grants that resulted from USD 
proposals that were scaled down by NSF. 

It was further provided, in fiscal year 1969, that universities that had 
demonstrated sufficient success under the first DSD award could apply for a 
second, and sometimes a third, DSD grant for new departments. 

According to William V. Consolazio,^ in his report on the subprogram 
for fiscal year 1%9; "This change was designed primarily to encourage 
each institution with the potential for significant growth and development 
to progress at a rate commensurate with its own needs and resources." 

In its early days, the dsd subprogram was also seen as a way of 
significantly increasing the number of awards made to the social sciences. 
Consolazio comments on this issue: "With respect to strengthening . . . 
science, it becomes increasingly clear that the social sciences are 
candidates for some special national effort and ihat the development ^ 
technique seems to be tailor-made for this purpose. He goes on to argue 
for the creation of a new special program for the development of the social 
sciences, but such a program was never brought into existence under 
^Science Development. 

A final innovative characteristic of the DSD subprogram was the manner 
in which DSD funds were used to encourage the development of interdisci- 
plinary institutes, programs, and so forth. 

* William V. Consolazio. ''Departmental Science Development/' \t\ Annual Report. Institu* 
lionai Support Programs for the Fiscal Year 1969 (Washington. D.C.: National Science 
Foundation, 1970). p. 42. 
^/bid., p. 50. 
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Yet another subprogram, the College Science Improvement Program 
(COSIP), was created under Science Development in !%7. Because COSIP 
was aimed at undeiigraduate education only, it is not part of this evaluation, 
J which ha^ focused exclusively on federal support for graduate education. 

College Science Improvement Program, in fact, was the subject of a 
previous evaluation study conducted at the American Council on 
Education/ 

These four programs— USD, DSD, SSD, and cosiP— constituted the 
Science Development institutional funding program. During the same 
period, NSF administered two different, but related, institutional support 
r- programs: Institutional Grants for Science and the Graduate Science 

J Facilities Prggram. Under the former,' relatively small formula grants were 

given to a large number of universities; the amount awarded to each 
institution was calculated as a function of the other federal support for 
science it received. Institutional Grants for Science differed from Science 
Development in several ways: the small size of each award, the laiige num- 
ber of awards given, and the use of a formula ^o determine amounts of 
awards, with no other attempt at systematic evaluation of an institution's 
science strength. As its name implies, the Graduate Science Facilities Pro* 
gram was limited to providing funds — matching grants, in particular — to aid 
in constructing n^w science buildings. 



SaECTION OF RECiPIENTS 

Typically, informal discussions between the Foundation and each potential 
applicant institution prececled the development of a formal proposal by the 
institution. Page reports that well over 200 informal inquiries were received 
shortly after the announcement of the program.' Once the institution 
submitted a formal proposal, the Foundation began a lengthy evaluation, 
which relied in great part on peer review— that is, review by scientists and 
scholars outside the Foundation — as well as on internal review. The first 
step was soliciting the opinions of outside experts about the proposal itself. 
Once this hurdle had been cleared, the Foundation sent a site-visit team to 
spend several days at the institution. Typically, this team comprised not 
only Foundation officials but also several outstanding scientists in the 
fields being proposed for funding and one or more experienced university 
administrators. After the site visit, each member submitted a lengthy report 
evaluating the proposal, the science capabilities of the university, and the 

• David E. Drew, On the Allocation of Federal Funds for Science Education, ace Research 
Reports, Vol. 5, No. 7 (Washington, D.C.: American Council on Education, 1970); A Study of 
the NSF College Science Improvement Program, ace Research Reports, Vol. 6, No,^4 
(Washington, D.C.: American Council on Education, 1971). 
' Page, op, cit. p. lOo. 
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overa]{ administrative strength of the schooK Further internal NSF adnunis- 
trative review was followed by a recommendation to a special panel of 
experts that advised the Foundation on Science Development. If this group 
* recommended that a USD proposal be approved, the school typically 
received about $4 million over a thre^e-year period. 

At the completion of the initial award, each USD recipient was 
encouraged to apply for supplementary funding to cover an additional two 
years. They were expected to submit a full formal proposal documenting 
their plans for those two years and including a statement of progress under 
the initial years of USD support. The evaluation procedures for the 
supplementary proposal rlosely paralleled those applied to the original 
proposal. Of the 31 USD recipients, 30 received a supplementary extension 
of fiinds. Thus, the typical USD recipient was awarded $6 million over a 
five-year period. 

In both proposals, the institution ivas expected to indicate (1) a clearly 
defined development scheme for the sciences that was part of a larger 
institutional plan, (2) the sources of university support thai would 
complement the funds being provided by the Foundation, and (3) plans for 
the continuation of the momentum provided by the proposed funding. 

Since SSD grants resulted from USD proposals, the procedures followed 
in evaluating those applications were identical except that s^^pplcmentary 
funds were not extended to those institutions. 

Departmental Science Development proposals were evaluated in much 
the same way as. proposals for the larger grants, but in a scaled-down 
manner. In the annual report for fiscal year 1967, the DSD evaluation 
procedures are described as **qui^ comprehensive," involving **a gicat 
deal of time and effort on the part of many people": 

Serious consideration is given to each proposal in order to assure that sound judgment is 
behind each recommended action. Many factors arc involvcdin evaluating proposals, such as: 
soundness and feasibility of the proposed development plan; commitment of the institution 
and its leadership to the plan; quality of the department leadership and its dedication to the 
plan; nature and quality of present faculty, students, and the department's research programs; 
adequacy of [the] department's facilities and equipment; quality of related departments, 
I>articu1ariy those associated with or impinging on tb^ plan; intellectual, financial, and regional 
Civility for attraction of high-quaiity new faculty and students; and, sq?propriatencss and 
adequacy of the proposed budget and of the institution's financial participation in the plan plus 
its capability to sustain the development program.* 

A key factor in assessing Science Development is the set of criteria used 
to evaluate the original proposals; presumably these criteria bear some 
resemblance to those used in this evaluation study. Howard Page com- 
ments on this issue, citing a critic of the program, Saunders MacLane, who 
labeled it ** vague, because it has no objective criteria/' According to 



• Annual Report: Institutional Support Programs for the Fiscal Year 1967 (Washington, D.C.: 
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MacLane, **a center of excellence usually requires some tradition and cer- 
tainly requires a sufficient concentration (a ^critical mass*) of first-rate 
scientists.**^ Page goes on to say that **the Foundation knows that *ob- 
jective criteria* are inadequate for measuring excellence, a fact especially 
bothersome to scientists; nevertheless, they have managed to get over 
that hurdle and begun to think like humanists/*'^ 

In short, the process that the Foundation used to evaluate proposals was 
aimed at excluding those universities felt to lack a sufficient basis for 
developing excellence under funding. At the outset, those schools already 
regarded as exhibiting excellence were excluded, although all concerned 
were careful never to release a public list of this **top 20" group. (There is 
some indication that such a list was circulated internally at NSF). Perhaps a 
statement attributed to Alvin Weinberg best summarizes the distinction 
between the **top 20** and the **second-tier** institutions: **An excellent 
school is one I will send my son to« and a good schooPis one i will send my 
nephew to.** 

Certainly, the decisions that faced the Foundation in evaluating propos- 
als bear a great resemblance to the decisions faced by researchers in 
evaluating the program. In that vein, a comment reported at the conclusion 
of Page*s article is particularly opposite: 

The evaluation of the effects of development grants was recognized as important but difficult. 
(One official declared forcefully that the government and the research community were more 
interested in spending money than in carefully evaluating what had been accomplished by 
money already spent. ''Now, the government is spending $15 billion a year in rdtae 
(Research, Development, Training, and Evaluation) . . . (and) how much of that is going into 
an analysis of how our rdtae is done? Peanuts.** **If they knew what they were achieving/' 
it was suggested sarcastically, ''they might spend less and that is .an argument for not 
knowing.'')' Some marks of failure— such as a university president's moving on to another 
job after securing the grant or putting all of his free money into buildings rather than people- 
were more evident than signs of success. Progress was not, by itself, proof that it had been 
caused by the grant, as many school receiving grants were on the upswing and would have 
improved in any event. Furthermore , a good plan, piepared not just by the president's office 
(as was too common with the poorer plans) but by the genuine involvement of a broad group 
of faculty, might be as responsible for an institution's progress as the grant itself." 



SCIENCE DEVELOPMENT AWARDS 

For a full understanding of the analyses reported in subsequent chapters, 
the reader should be aware of which institutions received Scjence 

• Saunders MacLane* ''Leadership and Quality in Science,'* inBastc Research and National 
Goals, A Report to the House Committee on Science and Astronautics by the National 
Academy of Sciences. 8yth Congr. 1st sess (March 1965). p. 200. 

Page. op. cii. p. 109. 
''Ihid..pp. 118-19. 
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Development grants, what kind of grant they received, and the amount of 
money involved.*^ Table 2- 1,2-2, and 2-3 present financial information for 
all recipients of USD, DSD, or SSD grants. For each recipient institution, the 
following information is given: the date of the initial award, total initial 
funds, the date of the supplementary award, total supplementary funds. 
For the DSD and SSD grants, the funded fields are listed. 

The dimensions of the funding program can be quickly grasped by 
inspecting these tables. Some additional summary data: in the USD sub- 
program the most heavily funded fields were physics ($50.3 million), chem- 
istry ($31.5 million), and mathematics ($1 1.2 million). Also, the breakdown 
of funds for personnel ($71.6 million), equipment ($65.5 million), and 
facilities ($39.9 million) is interesting. 

Financial data such as those in Table 2-1 give a complete, but rather dry, 
account of Science Development. A much richer and more interesting 
picture emerges when one reads the NSF files or visits the recipient 
institutions, as was done in this study. The funds have produced a number 
of exciting innovations. For example: 

Science Development permitted the University of Virginia to create a 
Center for Advanced Study in the Sciences through which they were able to 
hire both distinguished and promising younger scholars. The concept was 
so successful that they subsequently added other, nonscience fields (using 
other funds) and changed the name to the Center for Advanced Study. 

At Rice University scientists from many departments who had been 
working separately on problem^ requiring a systems approach were 
brought together in a new systems program under Science Development. 
The program drew upon economists, psychologists, electrical engineers, 
mathematicians, and so forth. 

At the University of Arizona, Science Development funds were used to 
bolster several physical science departments, notably astronomy. A major 
portion of the award to Arizona was used for the construction of a 90-in. 
reflector telescope on neighboring Kitt Peak. 

DSD funds were the catalytic factor in the development of an interdiscipli- 
nary, research-oriented Ecology Center at Utah Slate University. 

At the University of California, Santa Cruz, dsd funds made possible the 
development of a theoretical astrophysics thrust in the Astronomy 

Since the National Science Foundation considers the identities of institutions whose 
proposals were declined as confidential, no information on the declinations is given in this 
report. 
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TABLE 2*1 Un iyfttty Sd^nct D«v«lopm«nt RtdplMtt (funds in ttiouMnds of 
doMm) 





Initial Awanl 




Supptemcntary Award 


Institution 




AmAtint 


Date 


Amount 


Arizont 


07-02-65 


$4,045 


06-10^9 


$3,182 


Cvncgie-MjcUon 


05-10-67 


4.399 


05-26-72 


450 


Cftsc Wcsttm Reserve 






05-31-68 


2.160 


Case Institute 


05-13-65 


3.500 


— 


— 


Western Reserve 


05-13-65 


3.500 


— 


— 


Colorado 


06-30-65 


3.755 


03-31-70 


1.676 


Duke 


12-20^ 


2,527 


05-26-72 


650 


Florida 


07-02-65 


4.240 


02-07-69 


1,688 


Florida State 


03-21-68 


4,820 


05-26-72 


1.200 


Georgia 


08-29-67 


3.719 


03-31-70 


2,276 


Indiana 


12-20^ 


7.886 


^ 05-26-72 


1.470 


Iowa (Iowa City) 


08-29-67 


5.101 


05-20-71 


612 


Louisiana State 


05-31-68 


3.787 


03-31-70 


2.429 


Maryland 


05-10-67 


3.703 


05-20-71 


652 


Michifan State 


05-31-68 


4,307 


05-26-72 


1,180 


New Yorit University 


06-10^9 


4.560 


05-26-72 


1.600 


North Carolina 


05-10-67 


4.995 


05-20-71 


1,071 


North Carolina State 


05-19-66 


3.555 


05-20-71 


678 


Notre Danie 


05-10-67 


4.766 


05-20-71 


451 


Oregon 


03-05-70 


4,000 


12-31-69 


2.748 


Pittsburgh 


06-10-69 


3.650 


05-26-72 


800 


Polytechnic Institute, 


11-03-65 


3.332 


02-06-69 


1.210 


Brooklyn 










Purdue 


05-19-66 


3,600 


05-26-72 


300 


Rice 


06-30^5 


2,390 






Rochester 


06-30-65 


4.500 


12-31-69 


1,205 


Rutgers 


05-19-66 


3.708 


05-20-71 


334 


Texas 


05-20-71 


5,000 


05-20-71 


959 


Tulane 


05-20-71 


3.685 


05-20-71 


673 


use 


11-03-65 


4,473 


02-07-69 


3.000 


Vanderbilt 


05-10^7 


4.053 


05-26-72 


1.350 


Virginia 


06-30^5 


3,780 


06-10^9 


1,904 


Washington (Seattle) 


05-31-68 


5.000 


05-20-71 


906 


Washington (St. Louis) 


05-13-65 


3.919 . 


02-07-69 


3,090 



Department to complement the existing, applied orientation (based on the 
nearby location of the Lick Observatory). 

The Sociology Department of Washington State University used Science 
Development funds to develop rapid turn-around survey mechanisms and 
to create an urban field station for research. 
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TABU 2-2 SfMdal Sdtnc* Dtv«9opmmit Rtciplents (funds in thouMndt of doim) 



Inctifiitinn 


Date Amount Awarded 


Area 


Bnuideis 


03-19-70 $1,900 




chemistry, physics 




03-31-70 145 




environmental science 


Kansas State 


06-10-69 819 




biology 


Kentucky 


03-20-68 974 




mathematics 


Nebraska 


03-26-69 830 




chemistry 


New Mexico State 


04-12-66 700 




mathematics 


Northwestern 


05-19-70, 1,500 




urban systems engineering 


SUNY Stony Brook 


03-19-70 2,000 




astronomy and astrophysics 


Tennessee 


03-20-68 1,450 




chemical and metal engineering, 
physics 


Wayne State 


05-01-67 919 




chemistry 


West Viliginia 








University 


03-08-67 700 




engineering 


TABLE 2-3 D#|>«rtmtntal Sci«nc« D«v«lopnn«nt R«cipi«nts (funds in dollcrs) 


Institution 


Dale Amount Awarded Area 


Alaska 


04-30-70 $ 


7iU,0UU 


geology 


Arizona State 


03-26-69 


650,000 


solid state science 


Boston University 


03-05-70 


650,000 


biological science 


Bowling Green 


03-26-69 


531,900 


psychology 


Bryn Mawr 


05-23-69 


403,000 


biochemistry 


California, San Diego 


09-15-70 


571,000 


economics 


California, Santa Barbara 09-15-70 


480.000 


electrical engineering 


California, Santa Cruz 


03-26-69 


600,000 


astronomy 


Clare mont 


05-31-68 


491.500 


mathematics 




04-09-71 


466.000 


psychology 


Clark 


06-12-67 


545.070 


psychology 




03-26-69 


563.740 


geography 


Clarkson 


03-26-69 


800.000 


chemical engineering 


Clemson 


03-05-70 


650.000 


engineering 


Colorado School of Mines 04-30-70 


700.000 


geosci-mineral resources 


Colorado State 


^ t)3-05-70 


600.000 


civil engineering 


CUNY. City College 


05-31-68 


765.000 


physics 


CUNY. Hunter College 


05-31-68 


617.800 


biology 


Delaware . 


02-16.68 


556.000 


physics 


Denver ' 


05-31-68 


500.000 


mathematics 


Drexel Institute 


06-12-67 


527.700 


chemical engineering 


Emory 


09-15-70 " 


562.000 


chemistry 


Georgetown 


04-30-70 


460.000 


language and linguistics 


Hawaii 


04-30-70 


606.000 


chemistry 


Houston 


09-28-67 


420.000 


chemical engmeering 


Illinois. Chicago Circle 


05-31-68 


545.000 


chemistry 


Illinois Institute of 


03-26-69 


800.000 


biology 


Technology 








Kent State 


03-05-70 


400.000 


psychology 
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TABLE 2-3 Contimiwl 



Institution Date Amount Awarded Area 

3. 



Lehigh 


09-28-67 


550,000 


metallurgy and material 
sciences 




09-17-69 


670,000 


mechanical engineering 
and mechanics 


Louisiana State, New Orleans 


05-3j^ 


477,800 ^ 


chemistry 


Louisville 


05-31-68 


500,000 


psychology 


Marquette 


09-28-67 


540,000 


biology 


Massachusetts 


04-09-71 


582,000 


psychology 


Michigan Tech. 


03-26^9 . 


384,500 


engineering 


Mississippi 


12-29-69 " 


400.000 


electrical engineering 


Missouri 


09-28-67 


550,000 . 


physics 


Montana 


04-09-71 


500,000 


geology 


Nebraska 


03-26.69 


715,000 


physics 


New Hampshire 


03-05-70 


480.000 


psychology 


New Mexico 


06-12-67 


550.000 


mathematics 


Oakland 


05-31-68 


570,000 


engineering 


Ohio 


02-16-68 


563,000 


physics 


Oklahoma State 


03-26-69 


^5.200 


systems science 


Oregon State 


03-05-70 


600,000 


chemistry 


Rochester 


04-09-71 


848,000 


fundamental studies 
(multidisciptinary) 


RPI 


02-16-68 


569,000 


mathematics 




06-30-69 


490,000 


chemistry 


SMU 


02-16-68 


600,000 


electrical engineering 




03-05-70 


550.000 


economics 




04-09-71 


600.000 


anthropology 


South Carolina 


05-23-69 


500,000 


chemistry 


Stevens Institute 


09-17-69 


670,000 


physics 


SUNY, Albany 


03-26-69 


480,000 


mathematics 




09-15-70 


525,000 


biological sciences 


SUNY, Binghamton 


05-31-68 


500,000 


geology 




03-05-70 


390,000 


economics 


Tennessee Tech 


06-12-67 


300,000 


mechanical engineering 


Texas A&M 


03-26-69 


5iU\O00 


chemistry 




09-15-70 


458,000 


economics 


Tcx^s Tech 


09-15-70 


476,000 


electrical engineering 


Utah 


06-09-69 


720,000 


physics 




09-15-70 


695,000 


chemistry 


Utah State 


05-31-68 


550,000 


ecology 


VPI ^ 


09-17-69 


500.000 


geological sciences 


Washington State 


05-31-68 


550,000 


chemical physics 




09-15-70 


530,000 


sociology 


Wesleyan 


05-31-68 


560,000 


physics 


William and Mary 


03-05-70 


610,000 


physics 


Wisconsin, Milwaukee 


02-16-68 


550,000 


surface studies 


Wyoming 


02-16-68 


477.000 


geology 


Yale 


05-27-71 


1,500,000 


social sciences 


Yeshiva 


12-22-69 


900,000 


physics 
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Science Development funds to each of the three Triangle Universities in 
North Carolina help create a regional combined center of excellence. The 
funds enabled the University of North Carolina at Chapel Hill to become a 
leader in the quantitative social sciences. 



DENOUEMENT OF THE PROGRAM 

The impact and the image of Science Development is reflected in the 
following statement from a recent article about the National Science 
Foundation. 

Institutional support and feUowship grants . . . elevated . . . graduate schools and depart- 
ments to top rank. Privately, nsf was given much of the credit for the development of the 
highly regarded astronomy department at the University of Arizona, the mathematics 
department of Louisiana State University, and the physics departments at ftutgers and the 
University of Oregon, among others. Arizona alone received $7.2 million from nsf over the 
past five years.** 

f 

Talking about the Nixon administration's cutback of funds for higher 
education, Ihc same article goes on to quote Thomas Jones, President of the 
University of South Carolina: 

When reduction begins, [Jcncs] said, ** A ripple, then a wave, goes down through the pecking 
order of institutions as they scramble to find new sources of support. The lesser institutions 
experience an acceleration and disappearance of funds . . . and return to the second-hand, 
spectator-type science that characterized those institutions prior to Worid War II. 

*'One might conclude that these less than superb institutions are not really important to 
science in our nation," Jones said. **But let me say that nothing could be further from the 
tnith. The future economic health of our nation depends on at least one institution that is 
excellent in science in every state in our union.**** 

I 

The Science Development program was terminated by the Nixon 
administration in 1971 . The last initial USD awards were made in June 1969; 
the last supplementary awards were made in 1971, as were the last DSD 
awards. The final nine USD supplementary awards represented previously 
impounded funds that were relea;5ed by the Office of Managenient and 
Budget only after considerable lobbying by the institutions involved. Even 
at that, a total of only $9 million was awarded to those schools although, 
consistent with earlier supplementary grants, the initial Foundation 
recommendations had been considerably higher. 

Rudy Abramson, **Patron on the Potomac: The National Science Foundation,** Change* 
May-June 1971, p. 38. 
Ibid., p. 42. 
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SaENCE DEVELOPMENT IN HNANCIAL PERSPECTIVE 

As a glance at Table 2-1 to 2-3 indicates, the amounts awarded under 
Science Development were quite large by most standards; this report is 
devoted to examining the effects of the grants. It is reasonable to ask at the 
outset how these magnitudes compared with the total funds being awarded 
by the federal government for science to these and other institutions. Data 
on federal support for science per institution are compiled and published 
annually by the National Science Foundation through its Committee on 
Academic Science and Engineering. Table 2-4 shows the total federal 
science support per university for each year from 1963 to 1972 for each of 
two groups: the 31 USD recipients and a control group of comparable 
institutions." These same data are plotted graphically in Figure 2-1. 

An examination of this figure reveals a rather important point. While 
Science Development funds were large (i.e., millions of dollars), they 
represented only a small portion of the total federal aid for science that was 
directed at the recipient institutions during those years. As a result, there 
are many difficuhies involved in assessing the impact of Science Develop- 

»» For a detailed discussion of how the control group was selected, see Chapter 3. Appendix A 
lists all institutions in both the recipient and control groups (referred to there as "medium 
controls"). 
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mcnt funds. That is, in its peak years, the program was providing about $1 
or $2 million annually to each recipient institution, out of total federal 
awards to that school in the neighborhood of $15 million. Thus, even if 
these funds had significant effects, detecting them is quite a challenge in 
light of the huge amount of other forms of federal support. Conversely, any 
effects that are found will serve to indicate that the mechanism for directing 
and awarding these Science Development funds was particularly 
effective.** 

'* In addition to these problems, a related methodological challenge to an effective evaluation 
should be noted. An ever-increasing body of literature reporting research about the effects of 
various forms of intervention (e.g.. schooling) is coming to the conclusion that change 
strategies have no effect. In a sense, this is the general conclusion of such m^)or studies as the 
Colenun report (James S. Coleman er al.. Equality of Educational Opportunity (Washington, 
D.C.: U.S. Government Printing OfRce, I966)]r Christopher Jencks et al., inequality: A 
Reassessment of the Effect of Family and Schooling in America (New York: Basic Books. 
1972). and the work of Alexander W. Astin (see especially Astin. * 'Undergraduate 
Achievement and Institutional 'Excellence.* ** Science, 16 August 1968. pp. 66*~668). 
Essentially, all of these researchers used multivariate techniques to examine the effects of the 
educational environment while controlling for input: They found that when input was 
statistically controlled, the uniqueeffects of environment on output were minimal, if visible. A 
case can be made, however, that these conclusions are based on faculty methods: the lack of 
longitudinal data in one case, a blurring of input and environment in another, and so fdrth. 
That is,' while we shall be using multivariate techniques, such as regression, to test effects, we 
have tried to eliminate certain known hazards that tend to obscure impact. 
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Methodology 



The methodology for this research was carefully developed with the 
substantive questions of the study in mind. As indicated in Chapter 1» the 
focus upon quality graduate education implied the use of multiple criteria 
of excellence. Because a number of technical considerations dictated the 
need for longitudinal data, information covering a 15-year period from 
1958 through 1972 was used wherever possible.* This circumstance, 
together with the limited time frame in which the study was to be done, 
led to a heavy reliance on secondary analysis of existing data. As will be 
seen below, however, some additional data were collected during the 
course of the study. Another consideration was that any data used in the 
analyses had to be reliable and robust with respect to interinstitutional 
comparability. Finally, once the data were assembled, the selection of 
analytic techniques that would effectively untangle the causal relation- 
ships in the data was c.Mcial. Specifically, methods were needed to detect 
the Impact of Science >5velopment funds in universities with varying 
overall budgets and to differentiate Science Development effects from 
general trends taking place in most universities during this time. 

These considerations, then, constituted the frame of reference for 
developing a multifaceted research strategy that combined both qualita- 
tive and quantitative methods. 

> For a fuller discussion of the values of lomrifiidinal research, sec David E. Drew. ••The 
Potential Impact of Longitudi.ial Research on Decision Making in Higher Education." 
Journal of EducaiiGnal Data Processing, Vol. 9. No. 1-2. 1971. pp. 30-35. 
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SITE VISITS 



It was unnecessary to visit a// Science Development institutions to get 
maximum benefit from the field trips since th'^ diminishing returns would 
not havejustified the costs. Nine of the recipient USD schools, one recipient 
of a SSDgrant, and seven recipient DSD departments were visited.* These 
institutions were selected to include a variety of situations and thus to ex- 
plore differing ways in which the grants were administered, unusual institu- 
tional contexts, and so forth. 

- In addition, five Apnfunded institutions were visited for purposes of 
comparison. Given the qualitative nature of interviewing and the large 
number of variables operating, it would be pretentious to refer to these 
schools as "controls" in the rigorous statistical sense of that teim. They 
were carefully chosen, however, to provide a suitable contrast with the 
recipient institutions. The g(^al was to select **borderiine" institutions as 
controls; it was felt that >^the sc^hools most like the funded institutions were 
those schools that had applied^ for grants but had been rejected. . 

Other factors contributed to the selection of site-visit institutions. One 
was the objective of examining a range of grants from those thought to be 
successful to those thought to be failures. Both public and private 
institutions were included; in light of the NSF emphasis on regional 
dispersion of the funds, a deliberate attempt was made to achieve a 
regional dispersion of site visits. Table 3-1 summarizes the distribution of 
site-visit institutions, with respect to controi u>ublic or piivate), region, 
and type of grant. 

Since NSF considers the identities of institutions rejected for grants as 
confidential, it was decided early in the study not to reveal the names of 
any site-visit institutions in this report. This decision is consistent with the 
basic purposes of the study in that the site visits were used to generate 
general observations, conclusions, and recommendations; they were not 
intended as a school-by-school follow-up to assess how well grant funds 
had been spent. 

The range of grants studied during the site visits is indicated by the 
following list of selected programs funded at the site-visit institutions; 



Chemistry 
Physics 

Mechanic^! engineering 
Plant science 
Biology 
Microbiology 



Chemical engineering Psychometrics 



Mathematics 
Social science 
Geology 
Urban studies 
Information science 



Statistics 
Ecology 

Materials science 
Metallurgy 
Animal science 



* One institution that was visited received both a USD and a osoRrant, a fact that may explain 
some apparent discrepancies between the text and the accompanying tables. 
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TABLE'3-1 Typ« Control and Rogional Distribution of Sito-Vitit Institutions 



USD Control DSD and SSD Control Total 



Public 4 2^52^ 13 
Private 5 0 2^1 8 



Northeast 1 I 3 

Midwest I ^ 0 I 

'south 4 0 0 

Southwest 2 0 0 

West I 1 3 



Biochemistry 
Biophysics 
Behavioral science 
Astronomy 



Computer science 
Economics 
Political science 



0 5 

0 • 2 

t S 

0 2 

2 9 7 



Sociology 

Aerospace engineerihg 
Molecular biology 



The site-visit teams to Science Development schools and control 
institutions were ccmposed according to the following criteria: A member 
of the project staff, usually the director, was present on each visit. For 
visits to DSD institutions, the team was typically completed by one or two 
natural scientists who had participated in the preliminary NSF site visit 
during which the proposal was evaluated. The teams for the USD schools 
(and for controls) were larger, usually numbering four or five people. In 
addition to the project staff member, these teams commonly included one 
or more scientists who had been present on the initial visit to the school 
(or the subsequent visit when the institution applied for a supplementary 
extension of their funds). Moreover, each team included an educational 
administrator or a researcher with particular experience in higher educa- 
tion assessment. An attempt was made to include members of the study 
Advisory Panel or the National Board on Graduate Education on the 
site- visit I'cams. 

While scheduling was kept flexible in order to adapt to unique 
institutional arrangements, the typical schedule was as follows. Prior to 
each site visit, the project staff member thoroughly analyzed the NSF 
grant file on that institution. Tb5 teani met over breakfast on the day of 
the visit to discuss the particulSnssues to be explored. ^At an earlier date, 
each member of the team had been mailed information about the study 
and about the institution and its Science Development grant. In addition 
each had received a set of guidelines that provided the frame of reference 
for the visit. Soifte of the key questions-in those guidelines included: 

Ijr The overall goai of the University Science Development Progpram was to move 
universities into (he category of **centers of science excellence. ' Do you feel that this was 
accomplished here? 
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2. What organizational factors seemed to be most related to specific areas of success: 
e.g., those departments which profited most from the grant? What implications does this 
have for decision making in future funding programs? 

3. What organizational factors kerned to be related to departmental or institution-wide 
' failures? How might this be avoided in future funding prograiiis? 

4. What unique assets and liabilities denveii from the fact that the funding was given in 
this form of institutional support? Would this same level of funding have been more effective 
if given as project grants, on the one hand, or formula institutional, grants on the other? 

5. Was the school well prepared for the end of the grant? Have they been able to 
maintain the pace set during the funding? (This has particular implications with respect to 
state support of public institutions.) 

6. Did the grant originate at the department level, or was the proposal initiated by the 
central administration? How is this related to success or failure? 

7. Were the funds administered and controlled through the central administrations or 
were they essentially handled by the departments? How did this affect the success or 
failure? 

8. What were the spillover effects on nonfunded departments? 

9. Which kind of funding seemed to yieki the greatest payoff: for personnel, equipment, 
or facilities? Do some of these yield short>term and some long-term payoffs? 

10. Have there been any spillover effects en undergraduate education at the university? 
U. What is the best way to build a quality science department? How has this been 

related in this institution to the receipt of the Science Development grant? How do the key 
factors vary depending on the size of the department? 

12. Would more funds via a Science Development grant to the institution have made a 
significantly greater impact, or is there a limit to how much money can be used effectively at 
a given time? 

Since several of these guidelines were inapplicable to DSD, SSD and control 
institutions, several additional questions, focusing on characteristics 
unique to those institutions, were explored in visits to those schools. 

The visit began with a morning session during which the full team met 
with the president or chancelFor of the institution, the institutional 
representative for the Science Development grant, and key provosts, 
deans, department chairpersons, and so forth. The goal was to discuss the 
overall impact of the grant on the instiviution with those people who had 
been central to the administration of the funds. In the afternoon the 
site-visit team dispersed, each team member visiting one or two depart- 
ments to explore these issues in greater depth. One or two nonfunded 
departments at each institution vere usually includedThese nonfunded 
departments fell into two categories: those departments that had been 
included in early planning for the grant but did not receive funding 
(because of a university decision, a Fot^ndation decision, or both) and 
departments that never had been considered for the Science [)evelopment 
grant but that might have experienced some spillover effects. During the 
visits to departments, department chairmen, faculty members, and 
graduate students were interviewed, and equipment and facilities ac- 
quired under Science Development support examined. 
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After the preliminary meeting, the site-visit team reconvened once or 
twice concluding with a wrap-up session at the end of the day. Siib* 
sequently , each member wrote a report on his/her observations emphasiz- 
ing the guideline questions and focusing on his/her own area of expertise. 

Each of the site visitors contributed much time and energy to this 
endeavor, and their work was extremely valuable to the successful 
completion of the study. Table 3-2 lists all those who joined a visit and 
prepared a report; their observations are reflected in Chapter 4. 



QUANTITATIVE ANALYSES 

The nuuor focus of this study was on a series of quantitative analyses. The 
following sections describe some of the basic decisions that were made 
about sampling, the use of longitudinal data, and the techniques of 
analysis. In addition, the key input and control variables are listed, as are 
the dimensions of science excellence that were examined. 

Sample Selection 

In any large-scale evaluation study, selection of the experimental and 
control institutions is crucial. Early in the development of the design for 
this research, the notion of selecting a relatively small group of experi- 
mental and control institutions (say, ten of each) was^ rejected. We 
decided, instead, that the sample should inc? 'dc all USD recipients, many 
SSD and DSD recipients, and a sizable group of control institutions. 
Because of the many substantive and methodological objections that 
could be raised about any group of 20 or 30 institutions selected as 
controls, it was decided that the sample should comprise the entire group 
of doctorate-producing universities rated by Roose and Andersen in 
1970.^ This pool of 130 institutions included all USD and SSD recipients, 65 
percent of the DSD recipients, and a substantial number of control 
institutions. 

The rationale for this decision should be spelled out. In 1964 Allan 
Cartter of the American Council on Education (ACE) conducted a study in 
which he rated, field by field, the major doctorate-producing universities 
in the country in terms of the quality of the doctoral program and of the 
faculty.^The methodology was essentially peer review via a questionnaire 
survey. At that time, 106 universities met Cartter's criteria for inclusion: 

* Kenneth D. Roose and Charles J. Andersen, A Rating of Graduate Programs 
(Washington, D.C..: American Council on Education, 1970). 

* Allan M. Cartter, An Assessment of Quality in Graduate Education (Washington, D.C.: 
American Council on Education, 1966). 
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TABLE 3*2 SH* Vititort 



Or. John Bardeen 
University of Illinois 

Dr. Dietrich H. Bodenstein 
University of Viriginia 

Dr. Robert B. Brode 

University of California, Berkeley 

Dr. James W. Butcher 
Michigan State University 

Dr. William Cook 
Colorado State College 

Dr. Joan G. Crcager 

National Board on Graduate Education 

Ms. Charlotte Davis 
Washington International CoMege 

Dr. Joseph Doob 
University of Illinois 
Dr. David Drew 

National Board on Graduate Education 

Ms. Elaine £I-Khawas 
University Research Corpor&tjon 

Dr. Hsu Fan 
Purdue University 

Ms. Ann Finkelstein 

University Research Corporation 

Dr. John Foster 
Hampshire College 

Dr. Robtnt Herman 
General Motors Corporation 

Dr. Joseph O. Herschfclder 
University of Wisconsin. Madison 

Pr. Walter Hibbard, Jr. 
Owen-Coming Fiberglas Corporation 

Ms. Margo Jackson 

National Board on Graduate Education 

Dr. Manon L. Jackson 
University of Wisconsin. Madison 

Dr. Robert M. Johnson 
Florida State University 

Dr. Lawrence Jones 
University of Michigan 

Dr. Charles V. Kidd 

Association of American Universities 



Dr. David Krogmann 
Purdue University 

Dr. Lyle Lanier 

American Council on Education 
Dr.- Hans Laufer 
University of Connecticut 

Dr. Louis Levin 
Texas Tech University 

Dr. Frederick Lindvall 

Retired, formerly with Deere & Co. 

Dr. Ernest A. Lynton 
.University of Massachusetts 

Dr. John M^or 

Massachusetts Institute of Technology 
Dr. Jobn Millett 

Acade'my for Educational Development 

Dt< Elliott Montroll 
University of Rochester 

^ Dr. Lincoln Moses 
y Stanford University 

/ Dr. Norman H. Nachtreib 
University of Chicago 

Dr. Glenn Peterson 
Memphis State University 

Dr. Frank Putnam 
Indiana University 

Dr. Benton Rabinovitch 
University of Washington 

Dr. M-jredith Runner 
University of Colorado 

Dr. Irving Shain 
University of Wisconsin 

Dr. Michael Useem 
Harvard University 

Dr. Stefan E. Warschawski 
University of California. San Diego 

Dr. James Warwick 
University of Colorado 

Dr. Heinz G. Wilsdorf 
University of Virginia 
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In view of the decision to survey the major graduate schools, the simplest device was to 
include' the institutions which formed the Council of Graduate Schools in the United States 
in 1961. To this group of 100, we added six universities which had granted 100 or more 
doctorates (spiead over three or more"" fields) tn the preceding decade. Thus, the 106 
institutions include every university which averaged ten doctorates a year in the 1953-62 
period.* 

In 1%9, ACE repeated this survey under the direction of Kenneth 
Roose and Charles Andersen; the number of institutions meeting the 
criteria had risen to 130. Since this was the most recent rigorous defmition 
of this population, since we planned to work with the Cartter and the 
Roose -Andersen ratings, and since the Science Development program 
was at its peak in 1969, using these 130 schools as a base sample seemed 
to provide an optimal pool.* The universities in that group are 'isted in 
Table 3-3 as they originally appeared in Appendix B of the Roose- 
Andersen volume. 

For the speciflc analyses reported in subsequent chapters, this total 
sample was subdividex^ further. Let us take, as an example, the field of 
mathematics. In some analyses (e.g., those relating to trends in Ph.D. 
production) the first group consists of all USD recipients that got funds for 
mathematics (among other fields). The second group consists of those 
institutions that received DSD or SSD grants for mathematics.^ (DSD and 

^Ibid. p. 10. 

* The names of certain schools created some confusion among the research staff, usually 
because the school had changed names during the 15*year period covered in the analyses. 
Discovering such ambiguities in midstream, we decided to remove tltese few institutions 
from all analyses. The institutions were the University of California at San Francisco 
(Medical Center). Montana State University, and the University of Northern Colorado. 
'There was an ongoing debate throughout the study over whether ihe Special Science 
Development recipients should be included v th the uso or the dsd groups in the analyses. 
Shortly before the final computer runs the decision was made to include them with the 
DSD group. Since they had been grouped with the usD recipients in the preliminary nins. this 
decision ted to some changes in the structure of the control groups. Thus, in the results 
reported in this volume, sso recipients in fields other thai^ the one under discussion have 
been included in the control groups where previously they hkd been excluded. Despite these 
changes in the experimental and control groups, the final analyses did not differ markedly 
from the preliminary runs. ' ^ 

The chief reason for the decision was thit, even though the ^D grants were awarded on 
the basis of applications for the usi) program, in actual practice ^ sso grants more closely 
resembled the dsd awards in the following ways: (t iTliey averagedaround $1 million, which 
is closer to the^ $600,000 leverage for the DSO awards than the $6 millidaAverage for the USD 
awards; (2) they involved on^y a few departcncnts (and. in fact, it is hard to differentiate 
some of the institutions receiving sso grants frO^ institutions receiving several dsd grants); 
(3) they ran for three yeai^ acd could not be exten^^ ^^t^ supplementary funds, in contrast 
to the USD grants that covered a five-year period, including a two-year supplementary grant; 
and (4) like the dsd grants unlike the usd gr^flts, they did not emphasize acquisition of 
equipment or construction of new facilities. 
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TABLE 3-3 Inf tKutiont R«t«d by Room and Andmon 



Addphi University (N.Y.) 
University of Alabama 
American University (D.C.) 
University of Arizona 
Arizona State University 
University of Arkansas 
Auburn University (Ala.) 
Baylor University (Tex.) 
Boston University (Mass.) 
Brandeis University (Mass.) 
Brigham Young University (Utah) 
Polytechnic Institute of Brooklyn (N.Y.) 
Brown University (R.l.) 
Bryn Mawr College (Pa.) 
State University of New York at Buffalo 
University of California, Berkeley 
University of California, Davis 
University of California, Los Angeles 
University of California, Riverside 
University of California, San Diego 
University of California, San Francisco 

Medical Center 
California Institute of Technology 
Carnegie-Mellon University (Pa.) 
Case Western Reserve University (Ohio) 
Catholic University of America (D.C.) 
University of Chicago (111.) 
University of Cincinnati (Ohio) 
Oaremont University Center (Calif.) 
Clark University (Mass.) 
University of Colorado 
Colorado State University 
Columbia University (N.Y.) 
University of Connecticut 
Cornell University (N.Y.) 
University of Delaware 
University of Denver (Colo.) 
Duke University (N.C.) 
Emory University (Ga.) 
University of Florida 
Florida State University 
Fordham University (N.Y.) 
George Peabody College for Teachers 

Crenn.) 

George Washington University (D.C) 
Georgetown University (D.C.) 
University of Georgia 
Georgia Institute of Technology 
Harvard University (Mass.) 



University of Maryland 
University of Massachusetts 
University of Miami (Fla.) 
University of Michigan 
Michigan State University 
University of Minnesota 
University of Mississippi 
University of Missouri 
Montana State University 
New York University 
University of Nebraska 
University of New Hampshire 
University of New Mexico 
New School for Social Research (N.Y.) 
University of North Carolina at Chapel 
Hill 

North Carolina State University at 

Raleigh 
University of North Dakota 
North Texas State University 
University of Northern Colorado 
Northwestern University (III.) 
University of Notre Dame (Ind.) 
Ohio University 
Ohio State University 
University of Oklahoma 
Oklahoma State University 
University of Oregon 
Oregon State University 
University of Pennsylvania 
Pennsylvania State University 
University of Pittsburgh (Pa.) 
Princeton University (N.J.) 
Purdue University (Ind.) 
Rensselaer Polytechnic Institute (N.Y.) 
Rice University (Texas) 
University of Rochester (N.Y.) . 
Rockefeller University (N.Y.) 
Rutgers University (N.J.) 
St. John's University (N.Y.) 
Saint Louis University (Mo.) 
University of South Carolina 
University cf Southern California 
Southern Illinois University 
Stanford University (Calif.) 
Stevens Institute of Technology (N.J.) 
Syracuse University (N.Y,) 
Temple University (Pa.) 
University of Tennessee 
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TABLE 3^ Continued 



Unlversitv of Hawaii 


University of Texas 


Univer^itv nf Hniiston (Texas) 


Texas Agricultural and Mechanical 


Universitv •nf Illinois 


University 


Illinois Institute of Tcchnolosv 


Texas Tech University 


InrltAiia llniversitv 


TuAs University (Mass.) 


Lawrence University Institute of Paper 


Tulane University (La ) 


Chemistrv (Wise ) 


University of Utah 


University of Iow& 


Utah State University 


Inu/a ^tAtf* llnivi^rsitv 


Vanderbilt University (Tenn.) 


Irthnc Monktnc llniversitv (Md \ 


University of Virginia 


1 Inii/fr^ttv nf ICsmo!)^ 


Virginia Polytechnic Institute 


l^*4n^< ^tattf* llniver^itv 


Washington University (Mo.) 


llniversitv of Ken tuck v 


University of Washington 


Lehigh University (Pa.) 


Washington State University 


Louisiana State University 


wayne aiaie univcrsuy vmicn.; 


University of Louisville (Ky.) 


West Virginia University 


Loyola University (III.) 


University of Wisconsin 


Massachusetts Institute of 


University of Wyoming 


Technology 


Yale University (Conn.) 




Yeshiva University (N.Y.) 



SSD recipients in fields other than mathematics were included in the 
analyses as control institutions.) The third, fourth, and fifth groups were 
control institutions. The "high controls" were those schools that ranked 
above the highest USD recipients in initial quality as defined by the Cartter 
rating of the quality of mathemaiics faculty, done in 1964, before any 
Science Development grants had been awarded. The "medium control" 
group were those institutions that were roughly equal to the USD recip- 
ients in initial quality; this group comes closest to the conventional 
notion of a control group The "low controls" were those institutions that 
fell well below most of ^he USD recipients in initial quality. Only those 
institutions that had functioning, doctorate-producing mathematics de- 
partments at the time of both the Cartter and the Roose-Andersen 
surveys were included in this group. To summarize, the five major groups 
of institutions used in the analyses, and reported on in subsequent 
chapters, are: 

Experimental groups: Control groups: 

USD recipients High controls 

DSD or SSD recipients Medium controls 

Low controls 

Two additional groups of control institutions were defined for a few of 
the analyses. One consists of those USD recipients that received funds 
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only in fields other than mathematics. The other comprises those institu- 
tions that had no mathematics department in 1964 (at the time of the 
Carttcr ratings) but had developed doctorate-producing mathematics 
departments at the time of the Roose -Andersen survey in 1969. Sor^e of 
those institutions, such as The Rockefeller University, had developed 
outstanding departments in that short period of time, and it seemed 
inappropriate to lump them with the other "low controls**: i.e., to define 
them arbitrarily as 'Mow'* simply because they had no mathematics 
department in 1964. 

In a few of the analyses, the entire institution is used as the unit of 
study. In most cases, however, precise analysis required a field-by-field 
examination of the factors at work; in those cases, the groups defined 
above were broken out. For purposes of completing and clarifying this 
example, the specific institutions included in the five basic groups for the 
field of mathematics are listed in Appendix A. Of course, the definition of 
these groups differs for other fields (physics, chemistry, etc.); thus, the 
sample definition for each field is also given in Appendix A. 

The Overall Science Quality Index 

The concept of quality graduate education was constarajy at the forefront 
as we made various methodological decisions. The central challenge was 
to develop operational definitions of a meaningful, but very vague, 
concept, quality. An additional problem was posed by the rather loose 
language used to describe institutional quality in the proposals creating 
the Science Development program. As noted eariier, the funding program 
was aimed at the 30 or so second-tier institutions that fell just below the 
top 20 in initial s(jience quality. This language implies an unrealistic 
uniformity within the institution. That is, as any seasoned observer of 
higher education xnows, a school with an outstanding mathematics 
department may iiave a poor chemistry department and a mediocre 
physics department; science quality is distributed unevenly across de- 
partments within the same institution. This was one of a number of 
reasons for conducting field-by-field analyses in this study. 

In ranking institutions on the basis of their Cartter rating in a given field 
(say, mathematics), we found that the USD recipients were distributed 
widely, the greatest concentration being in the 20-50 ranks. As a measure 
of the degree to which the actions of the nsf administrators matched their 
rhetoric, a combined physical science measure was constructed; this was 
simply a computation of an institution^ mean score as derived from the 
Cartter (quality of faculty) n -s in three fields: mathematics, physics, 
and chemistry. When the insutuaons were ranked on the basis of this 
measure (see Appendix A), the USD recipients were found to be clustered 
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more tightly within the 20-50 range. Thus, while intrainstitutional varia- 
bility in departmental quality necessitated a heavy reliance on field-by- 
field analysis, ihe^ specially constructed overall science quality index 
tended to substantiate the notion that the USD recipients came from that 



To assess the impact of intervention aimed at change in higher education, 
longitudinal data are required. At a minimum, having a pretest measure on 
the criterion allows a researcher to test the hypothesis that differences in 
the measure after treatment do not simply reflect pre-existing differences. 
In addition. Campbell,' among others, has argueS'that a substantial period 
of time prior to the launching of the change program should be covered so 
that trends in the variables among experimental and control institutions 
prior to the awarding of funds can be taken into account. Unfortunately, 
not all previous research on the educational process has used a longitudi- 
nal design. 

Perhaps the best-known, large-scale study of students is the Coleman 
report.* The impetus for this research was political. The 1964 Civil Rights 
Act required that the degree to which equal opportunity existed in the 
nation*s elementary and secondary schools be assessed, with the added 
stipulation that the assessment be completed within two years. Although 
the findings proved provocative and produced an explosion of discussion 
and review, the fallout from that explosion contained many criticisms of 
the study attributable to the time constraint. To finish the study in the 
allotted period, Coleman and his associates had to assess changes that 
took place between, say, the first grade and the twelfth grade by 
examining groups of first-graders and twelfth-graders in a cross-sectional 
paradigm. The only way to draw accurate inferences about changes that 
take place during those school years and about the influence of various 
factors on these changes is to study the same group of students over a 
12-year period. Obviously, Coleman and his associates, limited by 
possibly ill-considered legislative strictures, were unable to use a lon- 
gitudinal design. In other contexts, however, where the need to find 
answers to problems is not so pressing or so politically loaded, such a 
design can and should be employed. The results of such research will 
usually be much less vulnerable to criticism. 

• Donald T, Campbell. ••Reforms as Experimcnls/' American Psychologist, Vol. 24, 1969, 
pp. 409-29, 

• James S. Coleman et at,. Equality of Educational Opportunity (Washington, D.C.: U.S. 
Govcmmcnl Priming Ofilce, 1966). 
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As indicated earlier, our reliance in the present s^udy on longitudinal 
data implied a reliance on existing data banks. The^work reported in 
subsequent chapters was based on a careful selectiotj of data from a 
number of federal and other data files; in most cases thesfe provided trend 
information from 1958 through 1972. \ 

Selection of Analytic Techniques \ 

The specific methods used in the analyses will be described iri greater 
detail in the appropriate sections. However, some general comments are 
in order here. \ 

Obviously, the technique of analysis must vary with the nature oMhe 
question being explored. In some instances, simple descriptive statistics 
(e.g., percentages and means) were deemed satisfactory. Thus, fot 
example, in examining the growth in faculty size of funded departments, 
one might begin with a simple cross tabulation of size, before, versus size, 
after. The next step might be to add a key control variable, such as public 
versus private (institutions). The next obvious step is to add an appro- 
priate control group of comparable nonfundJu institutions in which the 
same analysis has been carried out. The result would be a four- 
dimensional cross tabulation. 

But these simple descriptive techniques are inadequate for probing 
causal relationships; more sophisticated multivariate methods are needed. 
For this project, we implemented a multiple regression package at the 
National Academy of Sciences computer facility, as well as several other 
multivariate techniques. 

MEASURES OF SaENCE QUALITY 

Since multiple criteria^ were usedio operationaiize the concept of science 
quality, it seems appropriate at this juncture to briefly describe each of 
those indicators. 

c I. Perhaps the major thrust of the changes initiated under the Science 
Development program involved additions and modifications to the facul- 
ties of the funded departments. Thus, a key analysis in this study involved 
tracing the trends in faculty size in funded anH control institutions. 

2. To give an additional dimension to the above analysis (which 
focused simply on faculty size), an indicator of the research productivity 
of thb faculty was included. Specifically, we conducted a field-by-field 
analysis of trends in publication rates by faculties of funded and of control 
departments in the most frequently cited journals in that field. 
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3. Several criteria focused on characteristics of graduate students. An 
initial analysis traced trends in total graduate enrollments and in first- 
year, full-time graduate enrollments. 

4. A special survey was conducted to gather data that allowed com- 
parison of the Graduate Record Examination scores of students entering 
funded and control institutions before and ^ter the NSF program. 

5. A similar comparison was made of the baccalaureate origins of these 
graduate students. 

6. Since the rate of Ph^.D. production is such a critical issue in national 
discussions of manpower needs (of Ph.D. unemployment and so forth), a 
very careful examination was made of the rate at which Ph.D.s were 
produced by funded and by control departments. 

7. The previous measure is quantitative; a qualitative dimension was 
added by examining the characteristics of the academic institutions at 
which these Ph.D.s were employed in their first full-time job. (In the case 
of those Ph.D.s employed outside academia, we looked at trends in 
salaries.) 

8. One goal of the funding program was institutional self-sufficiency in 
maintaining the momentum begun under the grant. In light of that concern 
and of the heavy emphasis on research productivity underiying the 
Science Development program, a key dependent variable was the amount 
of outside research upport attracted to the department after funding; 
again, funded institutions were compared with control institutions. 

INPUT AND CONTROL VARIABLES 

The above list is a partial indication of the criteria used as reflections of 
science quality. In examining changes in those dependent variables and in 
assessing how these changes were related to the funding program, the 
pretest or eariier measures of these same variables were essential. In 
addition, several other f iCtors had to be considered in assessing the 
effects of the Science Development program: 

1 . From a substantive and methodological point of view, it was vital to 
take into account whether the institution was public or private. 

2. A given sum of money (say, $6 million over a period of six years) 
will have a dramatically different impact on an institution with an overall 
budget of $20 million than on one with a much larger budget. While no 
total institutional budget data were available which were comparable from 
school to school, we were fortunate to obtain an indicator of the annual 
federal support for science to the institution which met this criterion. 

3. In examining items such as faculty publications, it was necessary to 
control for faculty size. 
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4. Similarly, in examining data on graduate student characteristics, it 
was necessary to control for enrollment size. 

DATA ABOUT GRANTS 

In addition to the input and output variables, extensive information about 
the grants was gathered and coded for use in the analyses. As will be seen, 
**dummy'* variables were constructed to indicate whether or not each 
institution received a USD, SSD, or DSD grant or was a nonrecipient. In 
addition, detailed financial information was included in the analyses^^n 
the amount of funds awarded under the categories of personnel, equip- 
ment, and facilities — thus allowing multivariate analyses of the unique 
impact of funds for each of these purposes. 
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4 Site Visits 



The site visits provided a rich opportunity to observe the effects of the 
Science Development grants in a real setting. By examining the changes in 
faculty, graduate student enrollments, research, and so forth, at several 
funded (and nonfunded) universities, it was possible to add flesh to the 
statistical skeleton articulated in the analyses reported in subsequent 
chapters. 

Moreover, these site visits can stand by themselves as a separate case 
study of the effects of Science Development. In this chapter, the general 
effects of the programs as observed in these visits are summarized, the side 
effects are discussed, and finally comments arc made on a number of issues 
that arose in connection with the program. The methods used in conducting 
the visits have been described in detail in Chapter 3. 

In addition, the methods used in drawing the conclusions reported in this 
chapter should be described. As indicatc.d in Chapter 3„,cach site visitor 
produced a report giving his/her evaluatioi^s, usually running from three to 
five pages. These reports typically focused on the visitor's area of expertise 
and assessed the impact of the grant in tirms of some of the guideline 
questions listed eariier. Following the completion of the last site visit, all 
reports— both by staff and by outside obsc^rvers— were subjected to a 
content analysis. This analysis provided a profile of the general areas of 
agreement and disagreement among the site visitors with respect to the USD 
recipients, DSD recipients, control schools, and so forth. That content 
analysis formed the background for the development of the general 
observations reported below. 
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GENERAL EFFECTS OF THE GRANTS 

After visiting a number of campuses, an observer is struck by certain 
obvious conclusions. The Science Development funds brought about 
nuuicrcus positive changes. In most cases they allowed the universities 
to take a giant step forward in their science capabilities. While some prob* 
lems and liabilities were associated with the grants, the benefits greatly 
outweighed those detriments. In short, it is difficult to give $230 million to 
universities and do them much harm. 

Though many universities showed significant improvement — in some 
cases a dramatic quantum leap — it would be inaccurate to say that there 
now are 50 centers of excellence where a decade ago there were only 20. 
More than the Science Development program is needed to create SO insti* 
tutions whose science excellence parallels that of Cal Tech and Berkeley. 

Most administrators candidly admitted that they could not have ab- 
sorbed much more money than the program provided in the five-year span. 
They would, however, ;:ave eagerly welcomed the continuation of funding 
at the same level oyer a longer time period. We heard numerous com- 
plaints from university officials about the en^tic pattern of federal 
funding. One wry to head off such dissatisfaction would be to establish 
programs on the basis of federal commitments for longer time periods; for 
instance, a program aimed at a ten-year funding period would have ensured 
greater acceleration in the sciences and would simultaneously have 
avoided some of the problems occurring when the crunch hit higher educa- 
tion. (These problenris are discussed below.) 

One statement. frequently heard on the site visits was: '*These funds 
allowed us to do things that no other source would have provided money 
for." It seems a unique experience for these institutions to be given vast 
^ sums of money targeted specifically toward their science departments but 
\at the same time to be granted discretionary flexibility, 
^^ometimes these funds were used to establish new institutes, programs, 
or D^ildings that otherwise might not have existed. Frequently, they pro- 
vided support for activities that would eventually have been launched any- 
way. But the key factor is that these Science I>evelopmcnt funds greatly 
accelerated the pace of development, allowing universities to reach a point 
by 1970 that they might not otherwise have attained until 1980. 

Another term continually used by the recipients of these funds was 
**catalyst." Clearly, these grants had a catalytic effect both on the devel- 
opment of the sciences and on the acquisition of additional funds. As noted 
earlier, NSF did not impose a technical matching requirement, but uni- 
versities were expected to make significant financial contributions them- 
selves. A number were able to use the grant as a basis for obtaining addi- 
tional monies. For example, at one private university we heard about a 

36 



meeting between the head of the university and the board of trustees before 
the school received a Science Development grant. The administrator indi- 
cated that an additional significant contribution from the university was 
required; by the end of the meeting, he had raised severai million dollars ^ 
from the board. In additiorjt^to this positive impact on the acquisition of 
**matching"fonds, the necessity for the university to maintain the pace 
once the funding ended proved to be catalytic in acquiring funds furtjier 
down the road. Some state universities, notably thosf that were judged— » 
more successful, found this requirement to be useful^nincreasing the level 
of their state support. ^ 

Thus, one can easily see a nifmber of tangible positive effects of the 
program: brilliant new faculty acquired* graduate enrollment increased, 
buildings constructed, a telescope built, an excitirig interdisciplinary insti- 
tute established, and so forth. In some institutions, the progress was in- 
credibly dramatic, particularly given the short time span involved, an^ 
some^might argue that the progress of these schools alone would justify 
the program — particulariy when compared with most federally funded 
activities. But some of Uie negative developments associated with the 
grants should be noted as well. 

Some have maintained that Science Pevelopment was "the right 
program at the wrong time." it was launched in the niid-l%Os, when all 
science activities were spiraling upward. Then in the late 1960s and early 
1970s, when institutions were typically taking up their side of the bargain, 
the financial squeeze hit higher education: Federal funds were reduced, 
and the nation faced an **oversupply" of Ph.Ds in many fields. Some 
Science Development institutions found themselves overextended in the 
sciences; they had made a heavy commitment to a few major science fields 
that, because of NSF requirements, they felt obliged to maintain, occasion- 
ally at the expense of other fields. For example, in the eyes of some 
site-visit observers, a well-known program in agriculture at a major rural 
public university suffered because of this commitment to the basic 
sciences. 

The problem seemed particularly acute for some public institutions that 
had extracted commitments from the state at the time the grant was 
awarded. Five or six years laicv^ as the funding from Science Development 
was ending, they found that the state commitment in those fields was set in 
a budget that was not growing and bften was shrinking. Thus, occasionally. 
Science Development distorted university support toward these funded 
fields when the crunch hit. It was interesting to note, though, that virtually 
all institutions involved made a sincere, and mostly successful, effort to 
live up to their commitment. 

Another area hit particularly hard by the financial crisis was that of 
graduate student support. Frequently graduate students were attracted to 
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the university by aid fi'om Science Development funding and by the 
assumption that other resources would continue their support when the 
grant ended. Under more stringent financial conditions, however, support 
for graduate students was one of the first things to go. 

A related criticism is that Science Development funded, and developed a 
commitment to, fields relevant in thetnid-1960s, to the detriment of such 
fields as energy research and ecology that developed in the late l%Os and 
early 1970s. Given the Foundation's flexibility in redistributing funds 
within a department, it might have been advisable to structure the program 
so that it would have been easier for an institution to reallocate funds 
among departments as time passed. 

Occasionally, the program had other negative consequences. In a few 
instances, the funds were given to institutions where a power struggle was 
under way , thus increasing the power of one or more of the participants and 
exacerbating the struggle. 

It should be pointed out that the Foundation has a well-established . 
policy of never interfering with the inner workings of a university. In light 
of this principle. Foundation officials have often deliberately stepped back 
from tricky situations and refused to take a stand. The irony, of course, is 
that any program that gives a university $6 million is by definition strongly 
affecting the inner workings of that institution. Inasense, that is one of the 
explicit goals of the program. So there is a basic contradiction involved 
here. The degree to which the Foundation — or any other federal agency — 
should become involved with the internal affairs of an institution is a 
problem that must be worked out independently in each specific case. 
Federal support of higher education institutions inevitably raises some 
thorny questions. Either of two extremes is equally distasteful : universities 
languishing without federal support vs. federal control of all university 
activities. Institutional aid is a mechanism to provide federal support while 
giving the institutions significant control over the funds. In contrast, 
project support substantially reduces the university's authority, in that the 
individually funded researcher can always threaten to take his money and 
go elsewhere. 

One of the major benefits of the Science Development program was that 
it forced many institutions to plan in an organized fashion, as was often 
pointed out by people in universities or departments that were declined 
support. One is tempted to propose a scheme whereby a new national 
institutional funding program — one requiring extensive planning by the 
university— was loudly and widely proclaimed. The notion would be that, 
although no such program existed, the proclamation would lead univer- 
sities into the kind of planning activity that would still yield immense 
payoffs for them. 
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ELEMENTS OF SUCCESS 

When we compare extremely successful grants with others less so, certain 
distinguishing characteristics stand out. Perhaps the key factor is the 
strength of the central administration: Grants tended to be most successful 
at universities that had a strong and dynamic leader before, during, and 
after the grant; this-was particularly true of USD grants. While this factor 
may be less important, even unimportant, in the case of project support, it 
is vital with respect to institutional ajd. 

Continuity in office is a point thaf^shquld be underscored. Some 
upheavals were observed when an institutional leader who had been 
instrumental in acquiring the grant left prematurely (from the point of view 
of Science Development). Thus, not only a strong leader, but one 
committed to the university as well, is a vital element of success.* 

Even though all concerned tried to use multiple objective criteria in 
awarding grants, it is clear in retrospect that one cannot underestimate the 
impact of a powerful personality. Whether a grant proposal originated with 
a department or with the central administration, one person usually played 
a driving role. For example, at one public institution, the chairman of the 
physics department prodded other department heads into writing their 
sections of the proposal. Thus, both in the preparation of proposals and in 
the successes (and occasional failures) of the grants, the presence, of a 
strong individual was central. 

It may be that the person receiving project support from the federal 
government becomes more committed to his research and less dependent 
on or committed to the institution, whereas the person who successfully 
administers a Science Development grant has a stronger vision of what the 
institutioti or department might become. The chief lesson to be learned 
from this comparison of successes and failures is that, in future funding 
prograriis, reliance should be placed on the person who has demonstrated 
commitment to the institut:on— not on one who will leave or who will favor 
a pet area. 

Another component of success— and one associated with strong central 
leadership--is the existence at the outset of an overall development plan 
for the university. For many of the successful schools, the creation of a 
Science Development proposal amounted to carving out a section of an 
existing plan prepared as the result extensive self-study; the Science 
Development funds contributed to an overall balanced effort. In addition, 

• It appears thai one by-product of the funding program was to elevate a number of natural 
scientists in the recipient departments to the higher levels of administration at their 
institutions. Occasionally, the head of the institution who greeted us on the site visits was a 
recently appointed scientist from a funded department. 

39' 



these schools tended to be the ones that were best managed in general and 
as a result were relatively strong financially. Perhaps it is not surprising 
that in successful instituMons that had these characteristics, comments 
about how the prc^ram had helped to strengthen nonfunded departments, 
to improve undergraduate education, and so forth, were most frequently 
heard. 



CENTERS OF EXCELLENCE 

When this program was launched, the key term— particularly in the 
political rhetoric — was centers of excellence, which was usually used in 
connection with the development of regional centers. The term assumed so 
much importance that an analysis of its implications is something more than 
a word game. 

Both the term itself and the notion of regional distribution of funds had a 
dramatic political appeal tLat helped to persuade Congress and the public to 
fund the program — an advantage not to be underestimated, since decisions 
about science take place in an administrative and political environment. 

The term also had a positive impact on the recipient institutions. People 
at the school and in the surrounding area were proud that their institution 
was slated to become a ''center of excellence." Frequently, this improve- 
ment in image helped to attract matching funds and funds for continuing 
support. 

On the other side of the coin, this phrase, particularly when combined 
with the notion of expanding the top 20 institutions to 50, may have resulted 
in the setting of goals too lofty for a program of this size. As noted earlier, 
while many institutions made m^or advances under this program, few 
appear to have caught up with the top 20. 

in recognition of some of the liabilities of the phrase, nsf officials 
responsible tbrScience Development, notably Dr. Louis Levin, abandoned 
it midway through the program's history. But because of its early frequent 
use, the term stuck as a label for Science Development. 

The notion of a regional center is attractive. An excellent university 
benefits the surrounding area; it is a provider of skilled manpower, a 
repository of knowledge, a source for consultation to business and 
industry. The top 20 institutions are, for the most part, clustered in the 
Northeast, the upper Midwest, and California. At first glance, the USD 
grants do not seem to have taken up the slack in the rest of the country in 
any uniform way. Some states received two or three grants, while many 
received none. Certainly, the awarding of three grants to the clustered 
universities in North Carolina helped establish a regional center of 
excellence among these combined "triangle*' universities. But in another 
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case it is difficult to see the value of a USD grant to an institution in a city that 
already had two outstanding universities. The grants in the Southwest and 
West tended to be distributed along the coast; except for one USD grant to 
an institution in Colorado, there was no funded institution (USD) in a vast 
region including the Plains or Rocky Mountain slates. 

However, many other factors besides regional impact played a role in the 
selection of funded institutions. In fact, there was a basic contradiction in 
the philosophy of Science Development prc^ram in that funding second- 
tier universities and disbursing the funds on a regional basis are somewhat 
contradictory. That is, the eligible universities were not distributed 
uniformly throughout the country. Thus, the pattern of the USD grants 
appears more reasonable when compared with the national pattern of 
eligible schools and of population centers. 



USD VS. DSD GRANTS 

The Departmental Science Development program was initiated after the 
primary USD program had been under way for some time. The major reason 
for its creation was to bolster selective departments in otherwise weak 
institutions, the notion being that these departments were ready to move to 
excellence while the institution as a whole was not. The range of subjects 
funded by the DSD grants was wider and more varied than in the USDgrants. 
Moreover, DSD grants were given not oftly to departments in public and 
private universities but also to some primarily undergraduate institutions. 

' In addition to the greater diversity represented m the departmental 

grants, this subprogram was an excellent vehicle for generating truly 
interdisciplinary efforts. The site-visit teams observed some very success- 

I ful institutes that had been made possible through DSD funds and were 
functioning superbly as interdisciplinary endeavors. It is intriguing to 
speculate why this subprogram was so effective at fostering interdisci- 
plinary work. Perhaps the larger size and the multidepartmental nature of 
the USD grants called for more administrative work and was thus less 
effective at stimulating interdisciplinary efforts. 

With respect to the acquisition and implementation of DSD grants, the 
importance of a strong, charismatic leader was agam evident. The site visits 
indicated that the more successful DSD grants developed under the 
direction of a particulariy innovative department chairman or scholar. 
Perhaps this factor is related to the interdisciplinary thrust.. That is, a 
charismatic leader may be able to create a new interdisciplinary effort 
through a dsd grant without becoming enmeshed in the bureaucratic 
interplay required by the larger USD grants. 
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Even though these funds were targeted for single departments, they still 
were considered by the Foundation, and by the recipient universities, to be 
institutional grants. In one notable case, they became the focal point of a 
dispute between a powerful department chairman and the central adminis- 
tration at a public institution. The Foundation, forced to take sides, felt that 
given the philosophy of the program, it had to support the central 
administration. 

One weakness of funding a single department, in contrast with the 
broader USD support, is that peripheral, related substantive areas are not 
strengthened at the same time. For example, the researchers in one basic 
science department at a public institution located in a remote area 
complained that though their department had made dramatic strides, the 
weakne'ss of related departm.ents created a probleiti; this was compounded 
by their isolated location, which made collaboration with researchers in 
nearby universities impossible. In future funding programs limited to a 
single department, federal agencies should consider setting aside additional 
funds to bolster related fields at the same school. 



EFFECTS ON NONFUNDED DEPARTMENTS 

Studying the impact of the Science Development grant on nonfunded 
departments requires splitting this category into two groups: those that 
applied for funds and were rejected and those that never applied. Rarely did 
the former group indicate, after the fact, that they felt their rejection had 
been a Foundation mistake; indeed, in some cases, the rejection led the 
university to recognize a department's weakness and to attempt on its own 
to strengthen that area in addition to those that had been funded. For 
example, in the case of one major private institution, the Foundation 
indicated its willingness to support all of the proposed departments except 
mathematics. Forced to acknowledge the dismal national image of its math 
department, the institution (which had a highly successful experience 
under the grant)decided to rebuild the department and, during the period of 
the USD grant, poured substantial funds of its own into mathematics. 
Perhaps the story of the changes in that department is best reflected in the 
(full-time equivalent) faculty size before, during, and after the grant: from 
1 1 to 4 and finally to 26. In short, the entire mathematics department was 
revamped as a direct result of the Foundation's decision that it did not 
possess the strength to move toward excellence under USD funding. 

As another example, a department chairman in a public university 
whose application had been turned down when several other departments 
received USD funding commented that it was the best thing that had ever 
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happened to hi3 department. From that time on, he was able to plead 
poverty in his applications to other federal agencies and shortly landed 
another large^grant that helped the department to advance. 

Nonfunded departments in recipient institutions that had never applied 
for a grant (e.g., the humanities) had mixed experiences. In some cases, the 
generally increased affluence, stature, and momentum of the university 
had a positive spillover effect. For example, the chairman of the sociology 
department at a public university that received funds in the physical 
sciences felt that he had benefited indirectly from the USD grant in several 
ways. For one thing, the university, having developed the physical 
sciences, was now turning its attention to the social sciences. For another, 
the school's computer capability, built in support of the physical sciences, 
was useful to sociology as well. In other cases, nonrecipient departments 
felt resentful; this was manifested in the occasional complaint that those in 
the funded departments were receiving exorbitant salaries. 



STRATEGIES FOR DEVELOPING STRONG DEPARTMENTS 

As a by-product of this research, we explored the methods used by 
department chairmen and university presidents to develop strong depart- 
ments. Certain patterns emerged during the site visits. 

Most department chairmen felt that the way to achieve visibility in the 
academic firmament was to hire a few superstars. Three strategies were 
used to fill these distinguished professorships. First, some chairmen hired 
outstanding researchers from other universities.^ Second, according to^ 
some observers, the department head hired a relatively mediocre re- 
searcher and labeled him a distinguished professor. Third, extremely able 
and promising junior faculty members were promoted and given all the 
support, equipment, and facilities they needed. 

It is clear that the optimal strategy for department development varies 
fromfield to field. The chairman of a funded molecular biology department 
commented that his philosophy had been to hire only promising junior 
people because this field was new and did not have as many established 
stars as did other fields. Certainly, in some of the natural sciences, the 
development of a strong department requires a heavy investment in 
equipment and facilities, whereas in the social sciences or humanities, the 
focus is much more upon people. 

^ One of the issues surrounding the Science Development program, analyzed quantitatively in 
Giapter 5. is the degree to which the development of faculties in the funded institutions meant 
••stealing" good people from the top 20 and» in fact» from the other Science Development 
institutions. 
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NOTES ON PEER REVIEW 



By structuring the site-visit teams as described in Chapter 3 and by in- 
cluding scientists who had made earlier visits to evaluate the proposals, 
this study continued the tradition of peer review. Thus, it is possible to 
comment on the adequacies of that systefh. The small size of the circle of 
scientists from which such groups are usually selected has been discussed 
by others; the existence of interiocking directorates and overlapping 
boards is indisputable. All the scientists who participated in the visits were 
able, hard-working, and cooperative, but as one staff member noted, 
virtually all were also white, male, and over 60, even though the study 
made a concerted effort to break that pattern. The risk in such a situa- 
tion, of course, is that the range of opinions and assessments is likely to 
be limited. 

One possible way to break the pattern is to provide opportunities for 
younger scientists to participate on site-visit teams in evaluating proposals, 
programs, and so forth, A junior "faculty member might <:ombine the 
necessary education and background with a freshness of perspective 
missing in the present situation. In this project and others, there is the risk 
that site visits will become occasions for polite, gentlemanly mutual praise. 
Many scientists and administrators — who often are meeting with col- 
leagues whom they have known for some time— seem reluctant to ask 
difficult or piercing questions: e,g., abotit the deleterious effects of the 
funding. 



OBSERVATIONS ABOUT NSF ADMINISTRATION OF THE PROGRAM 

These site visits also provide an excellent opportunity for Monday morning 
quarterbacking: That is, the staff and board members on this study, having 
no connection with the NSF program, could assess objectively the 
adequacy of the Foimdation's administration of this program.^ 

As part of the research for this study, the project director made a 
determined effort to discuss the issues involved with a number of former 
and current NSF officials who had been responsible for Science Develop- 
ment. One clear impression that emerged from these discussions was that 
these administrators were well chosen and are several cuts above those 
generally found in the government bureaucracy. Clearly, the Foundation 
viewed Science Development as a major innovative program and sought 
out the best possible scientists and administrators to manage it. 

* The procedures and strategies followed by NSf- m managing Science Dcvelopmenl were 
described in Chapter 2. 
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Specific aspects of the NSF management of Science Development 
deserve both high and low marks. 

Again and again in these visits, recipients of the funds paid great tribute 
to the flexibility of the NSF personnel in administering the grants. 
Repeatedly, one heard stories of how, when the crunch hit and the 
institution requested to transfer funds originally allocated for personnel 
into equipment, for example, or vice versa, their requests were approved 
with relative ease. Cleariy, the Foundation had decided that the notion of 
institutional support implied a willingness to let the institutions manage 
the funds in the way that seemed to them most intelligent . It was common to 
hear a university administrator praise the Science Development bureau- 
crats as the best and most flexible he had encountered in years of deah'ng 
with Washington. 

The perspicacity of the Foundation's judgments in making awards was 
noted both at funded and nonfunded institutions and departments. Rarely 
did it appear, in retrospect, that the Foundation should have given a grant 
to an institution that it had turned down; the same held true for 
departments. Almost inevitably, a reassessment of the judgment years 
later revealed that it had been on target, and university and departmental 
officials who had worked on the proposal would often admit as much to the 
site visitors spontaneously. 

On the negative side, there is some evidence of interdepartmental 
jealousies within the Foundation that may have lessened an institution's 
chances of getting funding through another program. Apparently, the 
reasoning was, 'They already got a big chunk of money from Science 
Development so we should distribute our funds elsewhere." A less 
explicit, but more bureaucratically meaningful, reason may be that the 
credit for progress probably would have gone to the larger Science 
Development grant; thus, there would have been less payoff to the 
managers of other departments. It is unclear how serious this problem was 
or what the solution is. Perhaps a greater emphasis on intraorganizational 
communication would help. 

A more serious problem, which seems obvious in retrospect, is that the 
Foundation did not require enough from^he recipient institutions in the 
way of evaluative reports on the. effects of the funding. Though a few 
universities spontaneously provided such reports, with department-by- 
department evaluations, most did not, and it is virtually impossible to 
assess the impact of these funds by examining the NSF file on the 
institution. The Foundation did require progress reports annually and at the 
end of the grant; some schools were asked to submit additional reports. But 
typically these amounted to a brief text followed by the curriculum vitae of 
faculty who had been acquired under the grant. The relative lack of feed- 
back from the funded universities about what was accomplished is un- 
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fortunate. Such feedback would have been extremely useful to NSF and, 
in the absence of a large evaluative study such as this one, would have 
provided information about the progress of the grant. 

Dr. Louis Levin, a former key administrator of the program, noted in 
personal communications with the project director that the Foundation's 
aim was to minimize the amount of paperwork and administrative red tape 
for the recipient universities consistent with the general goals of the 
program. Nonetheless, requiring recipient faculty members and adminis- 
trators to pause and assess the effects of T e grant would have yielded a 
powerful payoff for relatively little effort on their part. Incidentally, Dr, 
Levin did initiate a survey of recipient institutions and departments that 
gathered detailed information on the distribution of funds and the number 
of personnel acquired under the grant, Whiie this survey stopped short of 
an extensive evaluation, it provided very useful data. 



MISCELLANEOUS OBSERVATIONS 

It is intriguing to speculate about why the social sciences were not more 
heavily funded in this program. As noted in Chapter 2, some saw the dsd 
subprogram as a potential vehicle for strengthening the social sciences. A 
former administrator of the program commented that the Foundation had 
tried without success to encourage more social scientists to apply for both 
USD and DSD grants. In fact, it encouraged the socia! scientists in one 
private university that already had received a USD grant in the natural 
sciences to submit a second USD proposal in their area. Just after that 
proposal was submitted, however, the entire Science Development pro- 
gram was halted. One wonders whether the natural science community 
would have been in abetter position to deal with the previously unforeseen 
environmental and energy crises — each of which has a social component — 
if the social sciences had been strengthened through the Science £>evelop- 
ment program. 

Finally, the major focus of this program was understandably on 
research. But the lack of concern about the student, especially the 
undergraduate, on the part of the Foundation officials, the faculty members 
at the institution, and so forth, is deplorable. Ironically, those institutions 
that expressed the most concern for undergraduate education were the 
same institutions that seem to have made the most dramatic progress in 
developing their science capabilities at the graduate level. 
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I \ 

5 Faculty Size 
and Mobility 



In the eyes of many people, developing a high-quality science department 
means, in essence, hiring excellent faculty members. Thus, it seems 
appropriate to begin discussion of the quantitative analyses with faculty 
attributes, though these constituted only one of several indices used to 
operationalize science quality. Clearly, the first step was to look at the 
changes in faculty size that resulted from Science Development funding. In 
this chapter, the growth trends of faculty in Science Development 
departments are compared with trends in the departments of control 
institutions and of the control department, history. 

An additional analysis was performed to see whether the changes 
observed represented a zero-sum or a non-zero-sum situation. That is,- 
Science Development and similar programs have occasionally been 
criticized on the grounds that, from a national point of view there is no net 
gain in faculty quality because Science Development institutions improve 
largely by "stealing'* talent from the top universities. The final section of 
this chapter reports on a special analysis conducted as a test of the 
hypotheses implicit in that criticism. 

DATA SOURCES 

Detailed information at the departmental level on university faculty is. 
unfortunately, not available for each year of the 15-year period used in this 
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study. Although such data have been collected for the National Faculty 
Directory during several recent years, data on the earlier years is lacking. 
The American Council on Education, in 1969 and again in 1972, conducted 
an extensive survey of a sample of the nation 's faculty, but this source, too, 
failed to provide a full longitudinal tracing of the demographic information 
sought. Finally, one or two professional societies— notably the American 
Institute of Physics— have gathered information on faculty in one field with 
some regularity. The information yielded by this source, however, would 
not have provided a means to compare trends across fields or, more to the 
point, between science and the control field of history. 

Thus, we sought a source that would cover time points from 1958 
through 1972, give full national coverage, and yield information on faculty 
at the department level for the three science fiekis and the control field. 

The data source that came closest to meeting all three criteria was the 
quadrennial American Universities and Colleges, a publication of the 
American Council on Education. This volume appeared four times during 
the 15-year period under consideration: 1958, 1962, 1966,and 1970. Itgives 
a detailed description of each college and university in the country, 
including a breakdown, by department, of the faculty size,* For each 
department, the faculty is further categorized according to academic rank: 
instructor, assistant professor, associate professor, full professor, part 
time, and other. ^ 



TRENDS IN FACULTY SIZE 

Before comparing faculty growth at funded and at control institutions, it is 
instructive to examine the genera! trends at all universities during the time 
period under consideration. Table 5-1 shows the departmental faculty sizes 

* The recent growth of branch campuses posed some problems in collecting these data. They 
were resolved as follows. In California and New York, it was usually easy to know which 
campus was meant In other states only the main campus (e.g.. the University of Wisconsin 
at Madison) was used for data on faculty size since it would be the one listed by Roose and 
Andersen as a m^jor doctorate-producing institution. Occasionally, in collecting information 
from other sources, we were unable to sort out branch campus data from data pertaining to the 
main campus, Thus, for instance, in retneving journal data from libranes. we encountered 
some periodicals that identified the author's afHliation as simply (forir'*'*p.ce) ' university of 
Wisconsin." Of necessity, such identifications were attnbuted to the main campus, in this 
example to the University of Wisconsin at Madison. 

* To code consistently the information from American Uni\ erstnes and Colleges, some minor 
technical adjustments had to be made A few institutions for which no data were reported had 
to be dropped from the analyses of faculty size. Occasionally, department names changed 
over the 15-year penod. Sometimes this was a change in designation only; in other cases, it 
reflected an actual change in the composition of the department, requmng extrapolation on 
our part to maintain consistency. 
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TABU S-1 Ov^niH Trtndt In Facutty Six*, by RtM (all maior inttHutionsh 





No. 


1958 


1%2 


1966 


1970 


Mathematics 


66 


21.9 


25.7 


36.7 


39.0 


Physics 


80 


17.9 


22.0 


28.7 


32.3 


Chemistry 


88 


18.3 


20.0 


23.7 


27.1 


History 


74 


15.9 


19.0 


24.9 


27.5 



* Includes all departments rated by Cartter in each (kid. except those USD instituuons not funded in that 
particular Held. 



for all institutions in each of the three science fields (physics, chemistry, 
and mathematics) and in the control field of history for the four time points 
for which data were available.^ The same data are presented graphically in 
Figure 5-1. Note that the general trend in each field was a sharp increase 
thoughout the l%Os, an increase that had begun to taper by 1970. 

Mathematics 

In Table 5-2, the trends in faculty size in the field of mathematics are 
presented separately for each of the five institutional groups: USD 
recipients, DSD and SSD recipients, high controls, medium controls, and 
low controls. In addition, these data are reported separately for public and 
for private institutions. Basically, faculty sizes at the USD recipients 
increased dramatically, particularly in comparison with faculty growth at 
the medium controls, the most appropriate control group. 

Further inspection reveals that this trend was evident in both the public 
and private sectors; in both sectors the USD recipients began with smaller 
faculties than the controls and finished with larger departments. Note that 
the mathematics departments in private institutions were much smaller 
than those departments in public institutions at each time point. 
Additional examination of the data revealed that the increase in USD- 
funded faculty sizes was most evident in those institutions with substantial 
overall federal science support, as contrasted with those with only 
moderate federal support for science/ 

' In this table, and all others in this cH iplcr. departmental size is computed by adding ail 
fulUtime faculty categories. 

* The distinction between "subslanliar* and 'moderate" science support was made on the 
basis of data published by the government on the annual total federal support for all science 
activities in each university. All institutions that were above the median in total federal 
science support were classified as having substantial support, all institutions below the 
median were classified as having moderate support, 
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TABLE S-2 D«partmtntal FicuHy Sizes in Science Development and Control 
Institutions: Mathematics 













1970 


All Inctttiitfnfic 












UoU iccipicnia 


15 


20.4 


24.3 


37.8 


42.2 


UaU rvCipicnis 


" 5 


IM 


14.2 


22.4 


33.4 


High controls 


17 




35.9 


52.7 


50.9 


Medium controls 


15 


24.2 


26 9 


37.2 


35.7 


Low controls 


16 


14.0 


17.9 


23.2 


28.8 


rUullC inMllUllUlls 














9 


22.0 


27.9 


44.3 


50.6 


r\Cf\ rM^tw\%0ntK 

u^u iccipiciiis 


2 


17.5 


18.0 


25.0 


43.5 


riign conirois 


o 


30.8 


47.7 


78.2 


74.2 


• iVicuiuin cunirui> 


tt 
o 


28.6 


32.1 


44.4 


43.9 


Low controls 


10 


19.0 


22.7 


29.7 


37.9 


Private Institutions 












USD recipients 


6 


18.0 


18.8 


28.0 


29.7 


DSD recipients 


3 


10.7 


117 


20.7 


26.7 


High controls 


11 


25.6 


27.1 


33.6 


33.4 


Medium controls 


7 


19.1 


21.0 


29.0 


26.4 


Low controls 


6 


5.7 


10.0 


12.3 


13.7 



Physics 

Table 5-3 presents the data on faculty size for the field of physics. The 
trends in the growth of physics faculties at public and at private universities 
differ strikingly. In the public institutions, the USD recipients began in 1958 
with significantly smaller faculties than the medium controls (18.1 versus 
25.3). By 1970, apparently as result of Science Development funding, the 
situation had been reversed; the USD faculties (42.9) exceeded the medium 
controls (35.4). In the private institutions (where, once again, departments 
tended to be smaller), the USD recipients started out with larger faculty 
sizes than the medium controls and retained this advantage. Thus, Science 
Development funding seems to have had no effect on faculty size at these 
institutions. 

Chemistry 

Tabic 5-4 shows the data on faculty size for chemistry . Once again, the USD 
recipients began with smaller faculty sizes than the medium controls; but 
after funding, the situation was reversed. Further examination of the table 
revealb, however, that this trend was most apparent at public institutions; 
in private institutions, the USD recipients had larger faculties than the 
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TABLE 5-3 D«p«rtm«nUi FacuKy Sizes in Science Development and Control 
institutiont: Physics 





No. 


1958 


1%2 


1966 


1970 


All Institutions 












USD recipients 




1 1 f\ 

1 /.D 






37.4 


DSD recipients 


o 


0 \ 


M 1 
1^. / 


17 7 
Iff 


7d 1 


High controls 


lA 

14 


w u 

^5 o 




A^ 1 


AH Q 


\.4(>fiiiim controls 


18 


19.4 


21 9 


28.3 


29.7 


Low controls 


19 


9.8 


tl.6 


15.8 


19.1 


Public Institutions 












USD recipients 


1^ 

ID 


IM 1 
1 0. 1 






42.9 


DSD recipients 


D 


0 

y.l 


11. 1 


17 ^ 

1 / .? 


^ / u 


High controls 


4 


\ 1 \ 


11 s 


Af\ n 


J J.J 


Medium controls 


8 


25.3 


27 6 


34.4 


35.4 


Low controls 


13 


II. 1 


13.5 


18.6 


22.6 


Private Institutions 












u.SD recipients 


9 


16.6 


21.6 


27.2 


27.8 


DSD recipients 


3 


97 


13.7 


18.0 


18.3 


High controls 


10 


35.3 


40.4 


45.0 


46 8 


Medium controls 


10 


14.7 


17 3 


23.5 


25.1 


Low controls 


6 


6.7 


7.3 


9.7 


11.3 


TABLE 5-4 Departmental Faculty Sizes in Science Dev^opment and Control 




Institutions: Chemistry 












No. 


1958 


l%2 


1966 


|y/0 


All Institutions 












USD recipients 


22 


18.0 


20.4 


26 5 


31 .3 


DSD recipients 


12 


16.2 


16 3 


18.8 


lA 1 


High controls 


15 


27 4 


28 9 


32.6 


IS 1 

J J.J 


Medium controls 


17 


21 7 


23.7 


26.9 


29.4 


Low controls 


24 


IL9 


13.4 


15 8 


18.0 


Public Institutions 












USD recipients 


13 


18.9 


21.6 


^0.0 


35.7 


DSD rccipirnts 


8 


18.9 


18.3 


21.1 


70/0 


High controls 


5 


35.0 


35.8 


40.4 


Ai\ f\. 


Medium controls 


13 


23.4 


25.2 


28.8 


31.5 


Low controls 


13 


13.5 


14.0 


17.8 


21.5 


I^vate Institutions 












USD recipients 


9 


16.8 


18.6 


21.4 


24.9 


DSi> recipients 


4 


10.8 


12.3 


14.0 


16 0 


High controls 


10 


22 0 


24 5 


27.8 


27.1 


Medium controls 


4 


160 


18 5 


20.8 


22.8 


Low controls 


II 


10.1 


11 6 


13 4 ' 


1.3.9 
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TABLE 5-5 Departmental FacuKy Sizes in Science Developnnent and Control 
institutions: History 





i\o. 


to<w 
lyjo 




1966 


1970 


All 1 P^Cfftf lit ft#«v^ C7 












USD recipients 


24 


15.5 


18.4 


24 4 


27 7 


High controls 


14 


28.3 


32.4 


41.3 


39.8 


Medium controls 


18 


14.3 


17.7 


22 <9 


26.7 


Low controls 


18 


9.8 


12.2 


16.8 


20.3 


Public Institutions 












USD recipients 


1 A 

14 


16.2 


19.5 


27.4 


32.2 


High controls 


4 


27.5 


30.8 


49.8 


54.3 


Medium controls 


10 


17.8 


21.5 


29.1 


33.8 


Low controls 


9 


8.4 


99 


16 7 


21.4 


Private Institutions 












USD recipients 


10 


14.6 


16.8 


20 2 


21.3 


High controls 


10 


28.8 


33.3 


37.1 


31.6 


Medium controls 


8 


10.0 


12.9 


15.1 


17.9 


Low controls 


9 


11 2 


14.6 


17.0 


19.2 



medium controls at each time point but one (where the two groups were 
about equal). ^ 



History 

Table 5-5 presents the data on the control field of history. Note that 
although the typical department grew, the five groups of institutions 
retained their relative position throughout the time period. In the analyses 
of data for this control field, al! USD recipients with rated history 
departments are included as the experimental group, but of course, no 
grants were given in history. In the private institutions, the trend differed 
slightly from that in the public institutions. The private USD recipients had 
somewhat larger history departments at each point in time than the 
controls, whereas in the public sector the situation was reversed. 



Summary 

Figures 5-2 to 5-5 graphically present the data on the USD recipients and the 
medium controls for all four fields. University Science Development 

* Note that the sample selection procedures described in Chapter 3 yielded only four private 
mstiturions in the medium control group for chemistry Thus, the data for this subgroup should 
be interpreted with extreme caution. 
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YEAR 

FIGURE 5-2 D«p»rtmental faculty sizes in science ckrvelopnwnt and control institution*: 
mathtmatics. 




YEAR 



FIGURE 5-3 Departmental faculty sizes in sciarv^ dftvelopment and control institutions: 
physics. 
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FIGURE 5 5 Dtp«rtm«nt«l faculty sizes in scitnc* d«veiopm«nt and control institutions: 
history: 
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recipients in all three science fields had smaller faculties than the controls 
prior to funding and larger faculties afterward. This marked improvement 
on the part of the recipients did not occur in the control field of history, 
where no grants were awarded. The significant growth of Science 
Development recipients was greater in the public sector than in the private 
domain. 

A MuKivariate Analysis of Impact 

The results presented above are essentially zero-order effects. They 
provide a basic descriptive profile of trends in faculty size in the 
experimental and control institutions without taking into account such 
related factors as total science funding to the institution. The next step was 
to examine the changes in these trends after these factors were controlled 
through use of a linear prediction model/ 

The following method was used for this multivariate analysis. A number 
of dimensions from 1959 to 1%1, which cleariy preceded the Science 
Development program, were selected to represent those factors that were 
most likely to be related to subsequent faculty size. These dimensions 
became the independent variables for a series of multiple regression 
equations: 

1. Total (departmental) graduate enrollment; 

2. E>epartmental publication rate; ^ 

3. Departmental doctorate production; 

4. Total federal science support for the institution. 

Where possible, the variables were created by taking the average of the 
values for these years. Since the earliest year for which data on federal 
support for science were available was 1%3, the figure for this year was 
used. 

The analytical technique was as follows: The independent variables, 
taken together, were seen as representing the status of a given department 
with respect to certain key dimensions prior to funding. These yariables 
were then systematically regressed on faculty size for 1%2, 1966, and 1970. 
In Table 5-6, the results from these regressions are presented. The table 
includes not only the multiple correlations but also the regression 
coefficients for each independent variable in each equation. 

Several mechanisms were used to test for, and isolate the effects of, the 
grant. First, a dummy variable was created, indicating whether or not a 

* In Ihis chapter, and in Chapters 6, 7, and 8, ihere will be slighl differences between the 
samples used in the regression analyses and those used m the zero-order plots because a few 
schools for which significanl data were missing had to be dropped from ihe former. 
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TABLE SS Prediction of Faculty SUm 









1970 


Maihemaucs {N = 65) 








iviuuipic corrciaiion 


0.64 


0.65 


0.59 


Rcffrcssion cocfTicients^ 






-0.13 


Do-:ioraie DFoduction 


0.18 


-0.06 


Piihlif^ation rates 


0.08 


0.27 


0.25 


Clll vlllllCllla 


0.41 


0.46 


0.42 


Qr>fpnr^ fiinHtno tn universitv 
Partial mrr^lafinnc^ 


0.05 


0.01 


O.IO 








Science development grant 


0.04 


0.12 


0.24 


All ff*Heral fiinH<i tn H^nartmpnt 


0.24 


0.35 


0.38 


Chemistry {N = 87) 








Multiple correlation 


ft 71 


0.72 


0.62 










r)<)ctorate Droduction 


0.12 


0.41 


0.05 


Piihlif^ation rates 


0.39 


0.09 


0.36 




0.26 


0.25 


0.29 


Q/«i«nr^ fiindino tn universitv 

^^iCIIvC luiiuiii^ \\f uiiivvi<9itjr 


-0.04 


-0.01 


-0.06 


rartlai cuiTCiaiiuiid 






0.30 


Science development grant 


0.06 


0,18 


AU federal funds to department 


-0,02 


0.06 


0.33 


Physics {N = 82) 






0.65 


iViuuipie corrciauuii 


0.78 


0 70 


Regression coefficients*^ 






-0.30 


Doctorate production 


0.10 


-0,08 


Puhiif^ntion rates 


0.21 


0.00 


0.28 


f< firntlmpnt<i 


0 42 


0,61 


0,42 


^f*t^nr>^ fiinilino tn untv^r^itv 
OClCnCC lUIIUIIIf^ lU uiiivvisiijr 


0,12 


0.21 


0.30 


ranicil corrciaiionb 






0.25 




0.06 


0.11 


All Torlpral fiinHc tn lipnarttnent 


0.20 


0,40 


0.20 


History {N = 71) 




0,76 




Multiple correlation 


0,75 


0.64 


Regression coefficients'* 




-0,26 


-0.27 


Doctorate production 


-0.33 


Publication rates 


0,69 


0.34 


0.21 


Enrollments 


0.43 


0 37 


0 34 


Science funding to university 


0,03 


0.38 


0.41 


Partial correlations* 






-0.01 


Science development grant 


-0.05 


-0.03 



* Regression coefficients are standardized 

• PartiaU controlled for the four variables above 
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department got a grant. Then the partial correlation of this variable with 
faculty size, the criterion, was computed while controlling for all of the 
predictor variables. The size of this partial in a given year reflects the 
variation in faculty size for that year that can be attributed uniquely to the 
receipt of the Science Development grant. Each of these partials is 
presented in Table 5<6. 

Data collected by the National Science Foundation yielded information 
about the total amount of federal support received by each department 
(including Science Development funds) in 1968. The partial correlations of 
this variable with faculty size also are included in Table 5-6. Differences 
between these measures and the grant partials provide a rough indication of 
the special impact of Science Development funds. [Comparison of these 
two partial correlations should be made with caution, however, since one 
of them (SD grant or not) actually is a partial point biserial correlation. 
That is, it is based upon a dichotomous variable, in contrast to the partial 
for departmental funds which is based upon a standard interval variable.] 

As a further mechanism for examining the effects of Science Develop- 
ment, the equations developed in this analysis were used to generate 
predicted faculty sizes for each department for each year. These **ex- 
pected" figures were then subtracted from the actual ^acuity size, yielding 
a residual score for each school for each year. Finally, ihese residual scores 
were averaged over the experimental and control groups and are plotted in 
Figure 5-6. 

In other words, use of this linear model allowed us to predict faculty size 
in eacjh school for each year on the basis of all key factors but Science 
Development funding. Differences in the predictive efficacy of the model 
between the experimental and control groups (as measured by the gap 
between these predicted rates and the actual faculty size) may be taken to 
indicate that the funding had an impact.^ 

This multivariate analysis strongly confirms that the Science Develop- 
ment program had a significant effect on recipient institutions, leading to an 
increase in the faculty sizes of the funded science departments. Both the 
partial correlations and the comparison of experimental and control 
residuals yield this conclusion. In all three fields, the growth in faculty size 
at the recipient institutions significantly outstrips what was predicted by 
the linear model based on conditions that existed prior to funding. For 
example, in 1970, the average USD chemistry department had 4.08 more 
members than expected, whereas the average medium control department 
had 035 more; in mathematics, there were 5.85 more faculty members than 
predicted at USD recipients, whereas the medium controls had 2.19 fewer 

' Lincoln Moses and Donald Campbell made significant and valuable contnbutions to the 
development of this methodology. 
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than predicted; finally, in physics, the comparable figures are 5.38 more at 
USD institutions and 0.82 fewer at the medium controls. 



ANALYSIS OF FACULTY MOBILITY 

Critics of Science Development, both before and after the program was 
launched, claimed that a funding program of this nature would not benefit 
the nation because it would simply support the movement of scholars from 
one sector of the academic world to another, the assumption being that 
outstanding scientists would be ''stolen" from the leading institutions by 
the USD recipients. These critics have claimed, for instance, that the only 
effect of Science Development on mathematics was to raise the salaries of 
the top 5 percent of mathematicians in the country. The question raised by 
such assertions is really whether Science Development funds had a 
non-zero-sum impact on the nation's pool of qualified scientists or 
whether, in fact, because of faculty movement, this was a zero-sum 
situation. This question is subject to empirical tests; that was the purpose of 
the analyses reported below. ^ 

Whatever such an analysis reveals, some basic philosophical issues are 
involved here. Implicit in the criticism is the assumption that the movement 
of a faculty member from one school to another does not represent a net 
gain for the nation. But many would argue that it is in the nation's interest to 
build up the science capacity of certain geographically dispersed univer- 
sities, even at the expense of the leading universities. The redistribution 
that results is not, in reality, a zero-sum situation but an overall advantage. 

On another level, one can question whether one department of 150 
physicists is equivalent in productive potential to three departments of 50 
physicists each. This line of thought evokes the concept of ''critical mass" 
as applied to faculty sizes: i.e., that a department must attain a certain 
' minimum size to function effectively but that, beyond a certain point, size 
yields diminishing returns. Presumably, the critical mass threshold varies 
from field to field. 

Having acknowledged these philosophical issues, however, the purpose 
of this section isjiot to debate them but to report on the empirical analyses 
that were conducted to assess the degree to which Science Development 



* These faculty mobility analyses were conducted under the initial sampling assumptions that 
grouped SSD recipients with usd recipients. Subsequently, as noted in Chapter 3, adecision 
was made to recategonze the ssd recipients with the dsd recipients. Time constraints pre- 
vented the reanalysis of these data in accordance with these final (minor) sampling modifica- 
tions. Consequently, the institutional sample sizes in this analysis differ slightly from those 
in other analyses reported in this volume. 
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faculties were developed by ''robbing" the leading institutions (or other 
Science Development schools). 

The analyses of faculty mobility were limited to one field, physics, for a 
variety of reasons. First, of all the fields funded by Science Development, 
physics received the largest amount of money. In addition, detailed data on 
departmental membership were available on a year-by-year basis for 
physics. As mentioned earlier, the American Institute of Physics regularly 
collects such information. 

The basic research design was as follows. The source school of each 
senior faculty member appointed to a USD school from 1965 to 1971 was 
identified. (Source school means the institution at which the faculty 
member previously held a position.) The patterns thus traced were then 
compared with the patterns for senior faculty members at the medium con- 
trol institutions. Finally, as a further control, the entire analysis was re- 
peated for the period 1959-1965. Each of these steps is described in more 
detail below. 

The notion of ''robbery" inherent in the criticism under investigation 
obviously applies only to senior faculty, notably "stars." Few would 
categorize the movement of a graduating Ph.D. to a junior faculty position 
at a Science Development institution as a case of "stealing." Con- 
sequently, of the total physics faculty at a Science Development or medium 
control institution, the "eligible" ones were those listed in the Directory of 
Physics and Astronomy Faculties as (1) chairman, (2) dean (or some other 
administrative title), (3) professor, (4) associate professor, (5) assistant 
professor, (6) on leave. Excluded were faculty members with the titles of 
(1) adjunct professor, (2) visiting professor, (3) lecturer, (4) instructor, (5) 
professor emeritus, (6) postdoctoral or research fellow. In addition to 
meeting our theoretical goals, this operational definition of senior faculty 
was practical in that all schools listed all our "included" titles in the 
directory, but not all of them listed the titles we were excluding. By 
examining the rosters of physics faculties at the USD physics recipients and 
medium controls, we were able to an'ive at a count of the "eligible.** i.e., 
senior, faculty. 

The next step was to determine how many of those eligible faculty 
members represented new appomtments during the period of Science 
Development funding. By comparing these 1971 (i.e., postfunding) lists 
with the l%5 (i.e., prefunding) directory, we were able to determine which 
of the eligible faculty represented new appointments at USD-funded and 
medium control institutions. (Because the comparison of the percentages 
of new appointments at USD and at control institutions was itself an 
interesting analysis, it is discussed below.) The sample under consideration 
now had been reduced to those faculty members representing new 
appointments in both experimental and control schools. The final sample 
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consisted of that subset of the new faculty members who were listed in 
American Men and Women of Science and whose previous jobs could 
therefore be identified.* Before we discuss the results of this analysis, a 
review of the terms used and their associated operational definitions may 
be useful: 

Eligible Faculty Senior physics faculty members listed as chairman, 
dean, professor, associate professor, assistant professor, on leave. 

New Appointments Those 1971 faculty members who were not at the 
same institution in 1%5, or those 1965 faculty members who were not at the 
same school in 1959. 

Experimental Schools Those institutions receiving USD or SSD grants 
for physics. 

Control Schools The medium control group of institutions for physics. 
Experimental Time Period 1%5-1971. 
Control Time Period 1959-l%5. 

Current Institution The institution at which the faculty member was 
employed in 197 1 (or in the case of the control time period, the institution at 
which the faculty member was employed in 1%5). 

Source School The institution that the faculty member left to take a 
position at the current institution. 

Table 5-7 presents data on the number of eligible faculty in 1959, 1%5, 
and 1971 and of these the number who were new appointments as of 1%5 
and 1971. The data are presented for all institutions, public institutions 
only, and private institutions only. By subtracting the 1%5 from the 1971 
faculty sizes (and the 1959 from the 1965 faculty sizes), we can obtain 
an indication of how many of these new appointments represented 
replacements and how many represented an increase in faculty size. These 
figures, as well, are included in Table 5-7. 

• Some minor technical decisions should be noted here Consistent with our reason for limiting 
the analysis to senior faculty members, the new faculty members included in the sample had to 
hold a senior position at both the old and the new institution. An exception to the above rule 
was made if the new faculty member was a Ph.D. who had worked for three or more years at 
his previous institution. This exception allowed us to include some experienced senior 
researchers who conducted their work in high-level professional positions that at some 
laboratones (c g. . the Plasma Physics Lab at Princeton) happened to carry a title like Research 
Associate. 

The control time pcnod (1959-1965) was constructed so that the interval (six years) would 
match that of the expcnmental time period. Our data collection procedures were such that we 
would miss scholars who had moved twice within either time frame ( 1960-1964. 1966- 1970). 
but who were at the same institution in a •'before** and an "after** year. 

The rare scholar who held a joint appointment at a recipient institution and at a source 
school was counted as one half in each category. 



63 



/ 



£ 

1 

Q. 
< 

z 



c 
a 
a 



vO rN 
^ ^* 



ON O 



M — 
d — 



c 

1 

a 
< 



v~j 50 O 
00 



f*^ 30 r«; 
m O O rsi 
m ^ f^J r*^ 



I 



Si 

So 



2 V3 



a 



rj ^ f*^ 30 



rr, >c 



64 



Some of the new appointments could not be located in American Men 
and Women of Science f presumably relatively junior scholars and foreign 
scientists. For example,, of the 443 new appointments to USD schools from 
1^5 f'> 1971, 94 were not listed. Of the remaining 349, 10^.5 had been 
$^nior faculty members at anot'-cr institution. Presumably, the other 240 
fejl into one of the following categories: (I) new/Ph.D.s, (2) foreign 
scholars, (3) researcli associates and postdoctoral fellows; or recruits from 
(4) private industry, (5) private or semiprivatt research organizations (e.g., 
Brookhaven and the Institute for Advanced Study), and (6) government. 

The central analysis, of course, involved comparing the source schools 
of the senior scientists who had moved to a target university (recipient or 
medium control institutions) Table 5-8 presents the results of that analysis. 
There, the origins of new appointees to both USD and control schools in 
both time periods are presented for ail institutions, as well as for public and 
private institutions only. The source schools were divided into seven 
categories: 

1. USD and SSD recipients in physics; 

2. Other USD recipients; 

3. DSD physics recipients 

4. High controls; 

5. Medium controls; 

6. Low controls; 

7. Other schools. 

The data in Tables 5-7 and 5-8 taken together present a complete picturie 
of the sources of all senior faculty who moved to usi> -^nd control 
institutions during each of the two time periods. In short, they reveal the 
mobility of senior faculty in physics at these institutions during the period 
of Science Development and during a comparable time period before the 
program was launched. 

Some obvious conclusions may be drawn. Prior to funding, USD and 
medium control schools were acquiring new senior faculty at about the 
same rate. After funding, the rate at the Science Development institutions 
remamed the same, while the rate at the medium controls dropped. In 
short. Science Development funds merely allowed the recipient institu- 
tions to maintain the pace they had set between 1959 and 1965, whereas the 
competition slowed do^n. 

These data do not support the criticism that the USD institutions 
developed their science capability at the cost of the high controls, or of any 
other group of institutions for that matter. If anything, the recipient 
institutions acquired faculty that might otherwise have gone to the medium 
controls — a reasonable outcome for such a funding program, flt is more 
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likely, given the trends in these data, that if the Science Development 
schools had not hired these new faculty members under funding, the 
control schools would not have acquired them either.) Most discussions of 
the "robbing" notion emphasize that the leading ii.Jtitutions lose top 
talent. These data reveal that, at least in the field of physics, they lost less 
senior talent overall during the funding period than before; the people 
leaving the leading institutions, however, were somewhat more likely to go 
to Science Development institutions than to their competitors. Given the 
large department sizes of the leading institutions, the defections in either 
time period were relatively trivial. 

Further examination of these data reveals that any advantage that the 
Science Development institutions may have eiyoyed over the controls in 
the ••after-funding" period was limited entirely to the public sector. With 
respect to acquiring new senior faculty, trends at the private recipient 
institutions more closely followed those at the medium controls than those 
at the public recipient institutions. The finding is intriguing inasmuch as 
Science Development funds per physics department for personnel were 
roughly the same in the public and in the private sector. 

The explanation for the advantage of public recipient institutions over 
other public institutions vvas suggested in the site visits, which revealed 
that public institutions used Science Development funds to hire new 
faculty, while at the same time extracting a commitment from the state to 
retain those faculty members once the funding ended. 

A more tantalizing question is why the private recipient institutions, 
having roughly the same amount of money per department for personnel as 
the public recipients did not grow at a greater rate during the period of 
funding. As Table 5-5 indicates, in 1971, the number of new appointments 
for expansion in the private recipients was 4,5, compared with 2.8 for the 
controls— hardly a striking difference. The explanation may be that during' 
the early 1970s, the recipient private institutions, faced with the financial 
crunch in higher education and finding it necessary to slow down their 
faculty growth, used thf ir Science Development money as replacement 
funds in the funded departments, diverting institutional funds into other 
departments. 

Taking a closer look at the high controls— i.e., the leading physics 
departments— we find that, in l%5, 40 of the % new appointments to USD 
schools came from these top departments, whereas after funding 57.5 of 
109.5 were drawn from this group. This represents a small increase of 
roughly one extra faculty member "stolen" from each high control school 
over a seven-year period. In contrast, the medium control institutions 
acquired 17 out of 36 new appointments from the leading institutions in 
I%5; in 1971 they acquired II out of 31. 

in short, the evidence refutes the criticism that the USD recipients 
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developed their science faculties by robbing the leading institutions. There 
is, however, one rather different way in which the leading physics 
departments may have si'fTered as a result of Science Development 
funding. It may be that the funding strengthened the recipient institutions to 
the point that they were less likely to experience defections. In short, the 
effect of the funding may have been to reduce the stealing of faculty 
members by the leading institutions. 



68 

ERLC 



6 Effects on 

Faculty Productivity 



However imporlanl an indicator of Science Development's impact the 
increased size of university science faculties may be, the constant focus 
upon quality— in the program itself and in this evaluative study— made ii 
necessary to look al the research productivity of these scientists,* The 
publication records of scientists from funded and from control institutions 
in the leading joumalj of their field was the measure of research 
productivity. This chapter reports the results of our analyses. Appendix B 
describes the methods used to select the key journals in each field. 

In recent years an increasing amount of the work done in the history and - 
sociology of science has centered onihe analysis of publication and citation 
rates. Publication rates are a straightforward index of productivity; 
citation rates indicate the impact of an author's publications on profes- 
sional colleagues. Studies of these two indices have yielded new knowledge 
not only about factors related to scientific productivity but also about 
patterns of communication within the sciences. 

In a 1966 assessment of the strengths and weaknesses of the standard 

» rhc other major function of a university professor— many would say the Jominant 
function— IS to teach . The unfortunate reality is. however, that there currently exists no valid 
measure of teaching performance that could be used to compare professors acrossjchools; 
I c. which has interinstitutionalcomparaDility In addition, of course, the philosophicafihrust 
of the Science Development program emphasized research productivity heavily For better or 
worse, relatively httle concern was given to teaching either at the graduate or the 
undergraduate level. 
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citation measure, Bayer and Folger=^ found that the measure correlated 
significantly with departmental prestige. Since then, Jonathan and Steven 
Cole^ have carried out a series of studies in this area, relying heavily on data 
produced by ? leading commercial firm, the Institute for Scientific 
Information, Inc., in Philadelphia; some of the analyses reported here also 
drew on data from ISI. 

In one notable study, the Coles^ argued that the citation pattern in 
science indicates that each field is dominated by a relatively small elite who 
make the major discoveries and who are frequently cited by others. This 
argument stands in contrast to the so-called Ortega hypothesis that 
progress in science is built on the efforts of a large number of lesser known 
researchers. Consistent with the Coles, Diana Crane maintains that 
scientific progress takes place in a social structure characterized by a small 
elite (the **invisible college'') that plays a key role in the communication of 
knowledge.^ 

Similarly, Derek DeSola Price has advanced the thesis that each 
scientific field has an "in'' group whose thinking and research dominate.^ 
His work has led him to conclude that **there is a reasonably good 
correlation between the eminence of a scientist and his productivity of 
papers/'^ In the same book, he discusses some of the implications of 
Lotka's findings that the rate of production of papers by authors is an 
inverse square function: That is, for every 100 scholars producing a single 
paper, 25 produce two papers, II produce three, and so forth. 

While these arguments are made in terms of individual scientists, not 
departments, they are pertinent to this assessment of a funding program 
based on the principle that the number of centers of science excellence 
should be radically increased. 

The short time that intervened between Science Development funding 

' Alan Bayer and Jom,. Folgcr. ' Some Correlates of a Citation Measure of Productivity in 
Science." Sociolo^n- ofkduiatum. Vol. 39 (1%6) pp. 381-390 

' Jonathan R. Cole and Stephen Cole. "Scientific Output and Recognition A Study in the 
Operation of the Re \\ard System in Science." >4m<'n(un 5<i( v/rn . Vol. 32(i%7) 

pp 377-390. Measuring the Quality of Sociological Research, Problems in the Use of the 
Science Citation Index." Amvruan Sonologist, Vol 6 (I97|) pp 23^30, ' The Ortega 
Hvp<nhesis."5< /f m Vol I78(l972)pp }bH-M*^,Sonal Stranfu atum in St u m t (Chicago 
University of Chicago Press. 1974) 

* Ci)le and Cole, op cit . 1972. 

Ouiiu Crane. Imtsihlt' CoUvycs Ihjfusion oj knoi\ U-Jfje //; Sm nttjU Communities 
(Chicago l/niversiiy of Chicago Press. 1972) 

• Derek J de Solla Price Donald deB Beaver. ' Collaboration in an Invisible College." 
Amvruan Ps\iholoHist.\o\ 2l(l*>66)pp. |()1U1018 

^ Derek J dc Solla Price, / ////<' .St u fu c. Bi^ Si wm i (New York Columbia University Press. 
196^) p 40 
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and this evaluation study effectively ruled out the use of citations as an 
impact measure, Citation rates were used, however, as the basic criterion 
for selecting the journals used in the productivity analyses. That is, in view 
of the focus on quality in this research, we felt it would be a mistake to 
analyze productivity simply by looking at publication in the full array of 
scientific journals; rather* it made more serise to look at publications in the 
leading, most often cited journals in the field. The fise of citation rates as a 
proxy for quality (whether of individuals, departrtients, or journals) is a 
notion that has been endorsed by the Coles, Kenneth Clark,* and Bayer and 
Folger.* While there are important technical differences between the 
methods used by this study and those employed by Inhaber, it should be 
noted that his study of physics journals also relied on citation rates 
provided by isi to select key journals.*^ Among the technical differences: 
In the current research, foreign language journals were eliminated, and 
multifield journals retained; Inhaber followed the opposite route. 

A number of researchers have found not only that the publication rates 
and the citation rates of scientists are closely correlated with each other but 
also that each is closely correlated with other measures of quality. For 
example* in a study relevant to this evaluation of Science Development, 
Hagstrom** used data drawn primarily from a questionnaire administered 
to a large sample of scientists to analyze the correlates of departmental 
excellence* as measured by Cartter. Hagstrom found that his publication 
measure (geometric mean per department) and his citation measure were 
about equally highly correlated with departmental quality, the correlations 
for publication being slightly higher than those for citations. 

After tracing the publication careers of a sample of scientists, the Coles 
concluded that there is a high correlation between citations and publica- 
tions. Those scientists whose early works are heavily cited tend sub- 
sequently to publish more than do their colleagues. According to the Coles, 
**these findings suggest that when a scientist's work is used by his 
colleagues he is encouraged to continue doing research and that when a 
scientist's work is ignored* his productivity will tail off."*^ They also noted 
that this correlation between quantity and quality is stronger in the nation's 
top departments than in the mediocre ones. 

" Kenneth O^k.Ameruan Fsychoio^ists. A Survey of a Growinfi Profession. (Washington, 
D.C.: Amencan Psychological Association, 1957). 
• Bayer and Folger, op. cit., 1966 

Herbert Inhaber, "Is There a Pecking Order in Physics," Physics Today. Vol. 27. May. 
1974. pp. 39-43. 

'* Warren Hagstrom, 'Inputs. Outputs, and the Prestige of Science Departments." 5rH/o/fj^y 
of Education, VoL 44 (1971) pp. 37.S-397. 
Cole and Cole, op. cit.. 1%7, p. 389. 
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Thus, I' growing body of empirical research on the sociology of science 
justifies the use of citation rates as a mechanisnri for selecting journals.'^ 
This same body of literature supports the notion that faculty productivity, 
as indicated by rate of publication in those same leading journals, can serve 
as a key indicator of quality, one that would correlate highly with numerous 
other measures of individual and departmental excellence. 

As indicated earlier, in studying faculty productivity , we faced a time-lag 
problem. That is, the impact of Science Development grants on citation 
rates could not be expected to become evident until some time after the 
study was completed. The most active years for initiating Science 
Development grants were 1%7 and 1968. Allowing at least a year for new 
faculty to be hired and several more years for work conducted under the 
grant to be written up and accepted for journal publication, it was just 
barely possible to detect effects upon publication rates; clearly, then, even 
more time is requiredfor a put' shed article to have an effect on the work of 
others, as reflected in citations to the article. Thus, it was necessary to look 
at publication rates rather than citation rates. But we did consider citation 
rates in selecting the journals to be used in the publication analyses. 

The basic plan in this productivity analysis was to compare faculty at 
funded institutions with those at control institutions in terms of their 
publication rates from 1958 to 1972. Consistent with our continuing 
emphasis upon quality, however, only the best journals in the field were 
considered. If the faculty at Science Development schools were found to 
be publishing more (or less) in these key journals, thi^s development would 
be much more significant than if they seemed to be gaining (or losing out) in 
relatively unknown, less selective journals. Moreover, restricting the set of 
journals to be analyzed is feasible in that the great preponderance of 
citations are made to these leading journals. For example, although the 
analyses for physics were limited to 20 journals, those journals accounted 
for 77 percent of all citations in the field. Finally, keeping the number of 
journals to be searched fairly small was a matter of practicality, given 
limitations of time and staff. 

Once the key journals in physics, chemistry, mathematics, and history 
had been selected, as described in Appendix B, the next task was to 
compare the rate at which faculty from funded and from control institutions 

Citation measures have sometimes been cnncized. usually on the following grounds they 
fail to differentiate between favorable and unfavorable citations. Frequently, researchers 
attempt to generate a "halo" effect by citing the major names in their field rather than the 
lesser-known people whose work may actually be more directly linked to their own Some 
, profound work that has had a pi)werful influence on all subsequent research is so taken for 
granted that scientists often do not bother to cite it. 

Use of the jsi citation file involves additional technical problems. For example, because of 
the structure of those files, work by two scientists with the same last name and first initial is 
indistinguishable 
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published in these journals during the period 1958 through 1972. The 
techniques involved both data from magnetic tape provided by isi and data 
retrieved clerically from journals in libraries. 

The IS! data were available for the three science fields for the period 
1%5-1972. The Institute for Scientific Information retrieves considerable 
information about the author, title, and so forth, of each journal article 
published in each significant science journal every year. From these basic 
data, several files are developed: a source file, a citation file, and the one 
most relevant to this study, a **corporate index" file, which records for 
each source article the institutional affiliation (or **corporate address") of 
the author. A corporate address is entered into the file just once for each 
article even if more than one of the authors came from that institution. The 
file is, of course, sorted by corporate address. Thus, by categorizing the 
corporate addresses in terms of the recipient and control groups for the 
study, we were able to count the total number of articles published in 
influential journals each year by faculty m each group of institutions. This 
count constituted our basic measure. 

As was pointed out earlier, the isi corporate address file covered only 
the 1%5-1972 period; earlier data (1958-1964) had to be collected by going 
through journals in libraries. To assure strict continuity, library retrieval 
was carried out according to rules conforming precisely to the somewhat 
idiosyncratic strategies used by isi.** 

A special problem in definition was posed by the wide variety of 
institutes and research centers with which scientific researchers are 
associated and which they give as their corporate address. Though specific 
decisions had to be made in each case, our basic criterion was as follows: If 
a research center was an integral part of a university, the university was 
"credited" with the article; those research centers which may have been 
in the same town as the university but were privately owned or not directly 
affiliated with ihe university were not counted.** 

Later, it was found that the two procedures differed in minor ways that would explain any 
slight gaps betwt;en publication rates i,. 1964 and in 1965. (How minor these differences were is 
reflected in the general continuity and smoothness of the curve from 1964 to l%5. in no case is 
there any sharp discontinuity ) Although isi never repeats a corporate address for an 
institution, the firm's definition of corporate address is somewhat unusual in that, if two 
researchers from different departments in the same institution publish an article, it will 
probably be listed under two corporate addresses, since frequently (but not always) isi 
includes the department as part of the corporate address. In those rare instances where this 
was the case, the library researchers would have credited the institution only once. Our 
belated discovery of these discrepancies led us to review extensively all corporate addresses 
generated from the tape files before we computed publication rates. 
'* After wc had made these decisions we checked the standard reference on this subject. 
Reseanh Centers Directory. 4th ed., and found that our decision had m each case 
corresponded to the classification in that volume. 
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OVERALL TRENDS IN PUBUCAT10NS 

As a framework for comparing publication trends in funded and in control 
institutions, it is instructive to look at the general trends within all major 
universities froml958 to 1972. Table 6-1 presents the mean departmental 
publication rates for each field over the period, and Figure 6-1 displays 
these same data graphically. The total sample consists of all Cartter-rated 
departments in each field, minus those USD schools thai had not received 
funds in that particular field. 

The average rate of publication in leading journals by these departments 
varied greatly from field to field. Thus, in 1972, the average number of 
publications per department was 79 in physics, 19 in chemistry, 54 in 
mathematics, and 3 in history. Publication rates in all three of the science 
fields, particularly physics, sharply increased over these 15 years, while 
the rate of publication in history remained about the same. Each of the 
science fields reached a peak in 1970 or 1971. In chemistry the 1972 rate 
was greater than the 1971 rate; in mathematics and physics, the 1972 rate 
was down from the earlier high point. 

There are several possible explanations for the slow down in publi- 
cation rates in science during the latter years. For instance, scientists 
outside the domain represented by these major universities may be 
publishing more frequently in the leading journals, thus leaving less space 
for the scientists from these schools. Or it could be that the relative rales 
remained the same but somewhat fewer articles were being published in 
1971 and 1972 than in 1969 and 1970. Further investigation is needed to 
reveal the answer to this intriguing question. 



COMPARISON OF PUBLICATION RATES IN SCIENCE DEVELOPMENT 
AND CONTROL INSTTTUTIONS 

A comparison of publication rates over time in funded and in control 
institutions reveals changes that show a clear effect of Science Develop- 
ment funds within several science fields but not in the control field of 
history. These results will be discussed on a field-by-field basis. 

Mathematics 

Table 6-2 presents the departmental publication rates per year in mathe- 
matics for all five groups of institutions. The entries in the body of the table 
represent the total number of articles published per department in all 20 
leading math journals combined. The results are presented separately for 
all institutions, for public institutions, and for private institutions. In 
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additipn. Figure 6-2 presents this information more dramatically for USD 
institutions and medium controls. As the table shows, the publication rates 
of departments in each group increased over time, except in the case of the 
high controls, which showed a slight decrease. The most dramatic increase 
occurred in the USD schools between 1968 and 1970, after the funding. It is 
conceivable that some of the loss shown by the high controls can be 
attributed to the transfer of faculty from those schools to the USD and other 
control schools. 

In examining these tables and graphs, one should bear in mind the time 
lag involved. Although the Science Development program was launched in 
1%5, many institutions did not receive funds until 1967 or 1968. Before the 
.efifects of the funding could be reflected in publication rates, time was 
needed for new faculty to be ^ired, for the faculty to produce papers based 
on their research, and for the journals to review and publish those papers. 
Presumably, in light of all these factors combined, one would first expect to 
see the effects of Science Development, funding in 1968-1970 (perhaps 
earlier if a faculty member listed his new institution on articles in press). 

A further examination of Table 6-2 reveals that the improved publication 
rate of the USD and medium control institutions, the widening gap be- 
tween them, and the drop in the publication rate of the high controls, were 
more pronounced in the public than in the private sector. 

Thus, it is clear that departmental publication rates in mathematics 
increased much more dramatically in funded than in control institutions. As 
we saw in Chapter 5, Science Development funds allowed the recipient 
institutions to expand their faculties considerably. Obviously, the next 
question is, ''Was ihp increase in publication rates simply a function of the 
larger staff size, or did the publication rate per person increase as well?" 
Table 6-3 presents data on. the publication rate per faculty member in a 
format analogous to that in Tablo 6-2. Bear in mind that, as noted in 
Chapter 5, data on faculty size were available for only four points in time. 
Thus, in the analyses reported below, the data have been extrapolated on 
the following basis^^ 

• 1958 data for 1958, 1959, I960; 

• 1962 data for 1%1 through 1964; 

• 1966 data for 1965 through 1968; 
We-xfata-for 1969 through 1972. 

Some minor discontinuities in the graphs (e.g., between 1968 and 1969) mpy 
be attributable in part to the shift in the basis for computing faculty sjze 
between those two years. 

As Table 6-3 indicates the publication rate per faculty member increased 
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much more in funded than in control institutions.** (Figure 6-3 presents 
graphically the trends in per faculty member publication rates for 
USD and for medium control institutions.) Once again, the changes were 
much sharper in the public than in the private sector. 

Finally, a comparison of the departmental and per person pub lication 
trends in institutions with substantial federal science support vs. those with 
only moderate support revealed that Science Development funding had a 
more dramatic effect on less affluent institutions. 

Physics 

Table 6-4 presents the departmental publication rates for the field of 
physics, including separate breakdowns for public and for private institu- 
tions. Figure 6-4 illustrates the trends in the USD and medium control 
groups. Though in the earlier years USD schools remained about equal to 
the controls in terms of publication rate, from 1970-1972 they surpassed the 
controls. The trends for the public and private sectors were about the same. 

Jn a comparison of publication trends in affluent and in less affluent 
institurions, the funding effect was apparent only in those institutions that 
received moderate federal science support. 

Examination of the data on per person productivity (Table 6-5, Figure 
6-5) reveals that the USD scientists consistently trailed below those from the 
medium controls until 1972, when the positions were reversed. In the 
private sector the changeover occurred earlier, in 1971. This small effect 
was seen in the moderately funded schools, but not in those receiving 
substantial federal science support. 

The high controls suffered no setback in physics. In the private sector 
especially, both the departmental publication rate and the individual 
publication rate continued to rise. 

Chemistry 

Table 6-6 presents the publication data for the field of chemistry, including 
separate breakdowns for public and for private institutions. Figure 6-6 
portrays tne ;e data for the USD schools and the medium controls. 

In comparing all USD and medium controls we find virtually no effect; in 
only one postfunding year did the former exceed the latter in publications 
noticeably. There does appear to be a slight effect in the public domain, 
however .1 

Note that these rates were computed by dividing the total number of pubhcations per group 
by thenumter of faculty members in thatgroup* this straightforward procedure ehminated the 
distortion jthat can be caused by differing departmental st/es when one averages rates 
computed ,'on a per department basis. 
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FIGURE 6-5 Publications per faculty member m science development and control 
institutions: physics. 
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\ FIGURE 6-6 Dtpartm«nUl publication ram in fcienc« davtlopmant and control inttitution* 
ch«n)istry. 




FIGURE 6-7 Publications p«r faculty mambar in Kianca rf*valopmant and control 
irtttitutiof»: diamrstiy. 
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There is no effect on per person publication in chemistry (except among 
those schools characterized by moderate amounts of total federal support 
for science where some impact of Science Development funding is evident) 
(Table 6-7, Figure 6-7). 

One might hypothesize that the difference between these findings and 
those for mathematics and physics reflect a greater lag time involved in 
chei^iistry publication. 

History 

As Figure 6-1 indicates, during the entire period from 1958 to 1972, the 
publication rate in history, the control field, was far below that in the three 
science fields. Moreover, there was no pattern of growth (Table 6-8, Figure 
6-8); indeed, in the 74 history departments considered, the per faculty 
publication rate declined (Table 6-9, Figure 6-9). Though funded institu- 
tions may have improved their standing slightly relative to control insti- 
tutions, that trend is by no means clear. 

A Multivariate Analysis of impact 

The results so far discussed are essentially zero-order effects, providing a 
descriptive profile of trends in publication rates in the experiijiental and 
control institutions. They do not take into account other factors (e.g., 
departmental graduate enrollments) that may influence these differences. 
The next step was to examine the changes in these trends when such related 
factors are controlled through use of a linear prediction model. 

The method used for this multivariate analysis was similar to that used in 
the study of changes in faculty size. A number of dimensions from a time 
period clearly prior to the initiation of the Science Development program 
(I959~1%1) were selected to represent those factors that were most likely 
to be related to subsequent rates of publication. This became the 
independent variable set for a series of multiple regression equations: 

1. Total (departmental) graduate enrollment; 

2. Departmental faculty size; 

3. Departmental doctorate production; 

4. Total federal science support for the institution. 

As before, the variables were created by taking the average of the values 
for the years I959-I%1 , where possible. In the case of faculty size only 

Even though wc collected 15 years* worth of longitudinal data, there were still not sufUcient 
trend data to use some of the statistical techniques developed by Box and Taio, and others, as 
a means of assessing the discontinuity in the curves. 
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those nmnbers for 1958 aiid 1962 were averaged. For federal support for 
science, clata for the year 1963 (the earliest available) were used. 

The independent variables tatoen-tqgether represent the status of a given 
department with respect to key related dimensions prior to funding. These 
variables were systematically regressed on publication rate for each year 
froin 1%5 through 1972. Table 6-10 presents, the results from these 
regressions and includes no,t only the multiple, conciliations but also the 
regression coefficients for e^ch independent variable in each equation. 

Several mechanisms were used to test for and isolate the grant effects. 
First, a dummy variable indicating whether or not a department received a 
grant was created. Then the partial correlation between this variable and 
publication rate (the criterion) was computed while all of the predictor 
variables were controlled. The size of this partial in a given year reflects the 
variation in publication rate for that year that can be attributed uniquely to 
the receipt of the Science Development grant. Each of these partials is 
presented in Table 6-10. 

Data collected by the National Science Foundation yielded information 
about the total amount of federal support received by each department in 
1968. The partial correlations of this variable with publication rate are also 
included in Table 6-10.*® The differences between these measures and the 
grant partials provide a rough indication of the special impact of Science 
Development funds. 

As a further mechanism for exaniining the effects of Science /Develop- 
ment the equations developed in this analysis were used to generate 
predicted publication rates for each department for each year. These 
**expected** figures were then subtracted (voni actual publication rates, 
yielding a residual score for each school for each year. Finally, these 
residual scores were averaged over the experimental and control groups 
(Figure 6-10). 

In other words, this linear model allowed us to predict publication rates 
in each school for each year on the basis of all key factors but Science 
Development funding. Differences in the' predictive efficacy of the model 
between the experimental and the control groups (as measured by the gap 
between these predicted rates and the actual number of publications 
produced) indicate that the funding had an impact on publication rates. 

Examination of the results of these analyses reveals an impact of Science 
Development funds on publications in each of the science fields but not in 
history, the control field. In all three science fields the USD and medium 

Partial correlations from these analyses of about 0.20or more are statistically significant. In 
particular, the threshhold values for czch field are mathematics. 0.21: physics. 0.18; 
chemistry. 0.18; and history. 0.20. 



92 



control residuals spread significantly as time passes: The USD productivity 
exceeds predictions,, while the control productivity falls below the pre- 
dicted levels. 

The reader should note that the residual scores provide a much better in- 
dication of impact than do the partial correlations. The former scores 
have been aggregated for each of the five groups; thus, it is possible to 
compare the USD trends with those of the .medium controls, the most 
meaningful control group. In contrast, construction of the dummy variable 
used in the partials required putting all controls (high, medium, and low) 
into one **nonfunded" category. As a result, direct comparison of funded 
departments with the medium controls is not possible with the partial 
correlations. 

As Table 6-10 and Figure 6-10 show, the unique impact of Science 
Development was manifested most dramatically in mathematics. Note that 
the partial correlatfons between Science Development funding and publica- 
tions for 1970, 1971, and 1972 were very high (for 1972, the figure was 0.33). 
Note also that the effect of these funds, as reflected in the partiais, greatly 
exceeds that of all federal funds to the department. That is. Science 
Development giants in particular had a marked efTect on publication by 
faculty in key mathematics journals. 

In chemistry the unique impact of Science Development funding was 
reflected in small positive partials that do not approach significance. The 
largest impact occurred in 1970. 

In physics, the partials were neither significant nor positive. In both 
chemistry and physics, the partials for all federal funds were much higher 
than those for Science Development funds alone. 

Table 6-1 1 and Figure 6-1 1 present comparable data from a multivariate 
analysis of the effects of funding on per person productivity. Residuals 
were calculated for each of the three years for which faulty size data were 
available. The only field m which these results indicate a possible Science 
Development impact is chemistry, where by 1970 the USD recipients were 
performing much better than the medium controls. 



Summary 

Taken together, the zero-order and multivariate results for total faculty 
productivity and per person productivity paint the following picture. 
Science Development funding had an impact cn departmental productivity 
in each of the three science fields. In chemistry this effect was revealed 
onlj. through the muho at iate analyses, but in math and physics it was clear 
from inspection of the zero order trends. The improvement in departmen- 
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FIGURE 6-10 Rtsidual analy«is of departmenul publicjition rat«s: (a) mathematics, 
(b) physics, (c) chemistry, (d) history. 
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TABLE 6-1 1 Prtdtction of Publicatloas p«r FiicuHy Memb«r 

1962 1966 1970 



^ Mathematics = 65) 
Multiple correlation 
Regression coefficients* 

Doctorate production 

Enrollment 

Science funding to university 
Partial correlations* 

Science development grant 
All federal funds to department 

Chrmistry (N = 87) 
Multiple correlation 
Regression coefficients* 

Doctorate production 

Enrollment 

Science funding to university 
Partial correlations* 

Science development grant 

Al^ federal funds to department 
Physics {N = 82) 
Multiple correlation 
Regression coefficients* 

Doctorate production 

Enrollment 

Science funding to university 
Partial correlations* 

Science development grant 
All federal funds to department 

History {N = 71) 
Multiple correlation 
Regression coefficients* 

Doctorate production 

£nrollment 

Science funding to university 
Partial correlations* 

Science development grant 



• Regression coefnciems standardiied. 

* Parttals controlled for the three variibles above. 



0.61 0 68 0.40 

0.10 0.32 0.01 

0.06 0.14 0.20 

0.48 0 30 0.22 

-0.01 -0.10 0.04 

0.14 -0.16 0.04 

0.61 0.62 0.64 

0.16 0.03 0.09 

0.28 0.33 . 0.18 

0.24 0.35 0.44 

0.04 0.04 0.02 

0.32 0.42 0.35 

0.75 0.83 0.55 

0.63 0.75 0.40 

-0.15 -0.14 -0.24 

0.27 0.22 0.35 

0.00 -0.05 0.09 

0.16 0.23 0.19 

0.52 0.36 0.41 

0.30 0.23 0.32 

-0.27 -0.36 -0.16 

0.45 0.35 0.25 

0.16 -0.02 O.H 



tal publication records at funded institutions was strongest in mathematics, 
definite in physics, and weakest in chemistry. The trend in mathematics 
was evident in both the public and private sectors. 

There was less impact on per person productivity. The math data 
seemed to indicate an effect, but the residual analyses raised a question as 
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to whether this effect was a function of Science Development funding or an 
artifact attributable to other factors. There seemed to be no effect in 
physics; the multivariate analyses revealed a possible impact in chemistry. 
No effects were seen in the control field of history. 

In short, recipient science departments clearly improved their publica- 
tion rates with the aid of Science Development funding. But the change was 
a departmental bne, probably the result of the simple increase in faculty 
size; the effects on per person productivity were minimal. 
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impact on 
Graduate Students 



Many would argue that the graduate students themselves are the essence of 
graduate education. Certainly » the future of scientific research lies with 
those who are currently undergoing this apprenticeship. We turn now to an 
assessment of the effects of Science Development on graduate enrollments 
and input characteristics; a study of the output— Ph.D. production- 
follows in the next chapter. Once again* the goal has been to combine a 
quantitative measure— enrollments — with an assessment of the impact of 
funding on quality* defmed in terms of the test scores and baccalaureate 
origins of entering graduate students. 



EFFECTS ON GRADUATE ENROLLMENTS 

To assess the impact of Science Development funds on graduate enroll- 
ments* we required a data source providing annual enrollment information 
that was comparable from field to field and from institution to institution. 
The source that came closest to meeting these criteria was Students 
Enrolled for Advanced Degrees -Institutional Data, an annual publication 
of the National Center for Educational Statistics (nces). Since the NCES did 
not begin collecting these data until 1959 and since the data for 1971 and 
1972 had not been published at the time of the study* our discussion of 
trends covers only this period (1959-1970)* rather than the full 15 years, 
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From these volumes were extracted two key indicators of enrollment 
irendsat all schools under study for the funded fields of physics, chemistry, 
and mathematics and for the control field of history: (1) the total graduate 
enrollment (men and women, full|ime and part time) and (2) first-year, 
full-time enrollment (men and women). Both indicators were used bec;^use 
we felt that^while it is imperative to examine trends in total enrollments, 
first-year trends were more likely to be sensitive to the impact of funding. 

Paut-time students were included in the former measure but excluded in 
the latter on the grounds that legitimate doctoral aspirants often enroll on a 
part-time basis while completing their dissertations; conversely, very few 
serious graduate students begin their studies on a part-time basis. 

Overall Enrollment Trends 

In Figure 7-1 trends in total enrollments for each of the four fields are 
plotted; Figure 7-2 depicts trends in first-year, fuil-time enrollments. These 
two curves reveal a good deal about the changing graduate student 
population between 1959 and 1970. Graduate studem enrollments in history 
tended to be larger than those in any of the three sciences. In each field, 
total graduate enrollments grew more or less steadily, peaking in 1967 and 
1968 and then declining; similarly, first-year, full-time enrollments in the 
sciences grew steadily until the mid-1960s, when they peaked. 

Against this background, it is interesting to compare the behavior of 
Science Development and of control departments. 

Trends in Science Development and Control institutions 

Table 7-1 presents average departmental enrollments in mathematics for 
each year from 1959 to 1970 in each of five groups of institutions: USD 
recipients, DSDand SSD recipients, high control, medium control, and low 
control. Data are presented for all institutions, public institutions only, and 
private institutions only. Figure 7-3 plots the total enrollments for the USD 
schools and the medium controls; While the **all institutions'' data reveal 
no impact of funding, separate analyses show a difference between the 
public and private sectors. In the private domain the medium controls ex- 
ceeded the USDS before, during, and after the grant. However in the public 
sector, there appears to have been an impact of Science Development fund- 
ing upon these total graduate enrollments. 

Table 7-2 presents data on average first-year, full-time departr^fiental 
enrollments in mathematics for the same five groups of institutions. In 
Figure 7-4 the data for the USD recipients and the medium control group 
have been plotted. There is no impact of the funding on first-year 
enrollments revealed in these data. 
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A comparison of total and first-year math enrollments in schools 
receiving substantial federal science support, as contrasted with those 
receiving moderate amounts of support, revealed an impact in the former 
sector. That is, among those schools that received considerable amounts of 
money for the sciences from all federal sources, the math enrollments at 
Science Development institutions grew more rapidly than those at control 
schools. This may indicate that Science Development funds alone are not 
sufTicient to motivate a school to expand enrollments but rather had to be 
part of a larger picture of relative affluence. ' 

Table 7-3 (total enrollments) and 7-4 (first-year enrollments) and Figure 
7-5 (total) and 7-6 (first year) present comparable data for physics. No 
effect of Science Development funding upon physics enrollments is seen in 
these tables. However, when substantially funded institutions are com- 
pared with moderately funded institutions an effect of Science Develop- 
ment funding on both total enrollments and first-year enrollment is seen in 
the wealthier schools. 

Table 7-5 and 7-6 and Figures 7-7 and 7-8 present the graduate enrollment 
data for chemistry . Once again, there appears to be no impact of funding on 
either total or firsf-year enrollments with the possible exception of the 
public sector. Here, the USD recipients began with total enrollments quite a 
bit below those of the medium control groups; by the end of the funding 
period, however, the two enrollments were about ecjual. 

Finally, Tables 7-7 and 7-8 and Figures 7-9 and 7-10 present graduate 
enrollment data for the control field of history. No relationship between 
Science £>evelopment funding and eitfier enrollment index is seen in 
history. First-year enrollments at the USD recipient institutions exceeded 
those at the medium controls throughout the entire time period. 

A Muhivariate Analysis of Impact 

The same multivariate tests of impact that were employed earlier with 
respect to faculty size and publication rates were applied to the data on 
enrollments. That is, a linear model, based on measures reflecting 
conditions at the institutions well before funding began « wa« lo 
estimate total (Table 7-9) and first-year (Table 7 10) graduate enrollments at 
Science Development institutions and at control institutions for each year 
between ^965 and 1972. The predictor variables, and the years to which 
they applied, were publication rate, 1959-1%!, faculty size, 1958, 1%2; 
.Ph.D. production, 1959-1%1; and total federal science funding. l%3. 

As before, a series of regression equations predicting enrollments for 
each year were generated. The total enrollment results are presented in 
Table 7-9, which includes each multiple correlation coefficient and several 
partial correlation coefficients: One partial correlation is between enroll- 
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FIGURE 7-6 Dapartmantjl anrdtmants in scianca davalopmant and control institbdofis 
(first-yaar« full-tima graduate students only): physics. 
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ment level and a dummy variable indicating whether or not the school got 
a grant; the second partial is between enrollment level and total depart- 
mental science funding (as measured in 1968). Both partials were computed 
after all predictor variables were controlled for. 

Each year a residual scor|&^as computed for each institution by 
subtracting the estimated eni^teient from the actual enrollment. These 
residuals, averaged for both th^xperimental and the control groups, are 
plotted in Figure 7-11. " 

In Table 7-10 and Figure 7-12 information on the results of a similar 
analyses with respect to first-year full-time enrollments is presented. 3 

The multivariate analysis leads to some interesting conclusions." In 
mathematics both the pattern of partial correlations and the residual plots 
indicate an effect of funding on first-yearenrollments. In physics this seems 
true for both total and first-year enrollments; however, the gap between the 
USD and control residual curves is seen as early as 1%5, an indication that 
factors other than the NSF program may be at work. The residual analysis 
shows no impact in either chemistry or history.* 

In short, while the zero-order analyses revealed little impact of the 
funding on either first-year or total graduate enrollments (with the ex- 
ception of public institutions in mathematics), the multivariate analyses 
show an impact on first-year enrollments in math and, possibly, on botlv, 
enrollment indices in physics. 



EFFECTS ON GRADUATE STUDENT QUALITY 

The analyses reported above, while necessary, dealt with ihcquantity, not 
the quality, of graduate students. Since Science Development aimed at 
improving the quality of science at the recipient institutions, also the focus 
of this e valuati ve study , additional analyses were conducted using two indi- 
cators of the quality of entering graduate students: (1) scores on the Grad- 
uate Record Examination (a standardized test of aptitude, with a verbal 
and a quantitative component) and (2) the selectivity levels of students' 
baccalaureate institutions. 

Obtaining information on GRE test scores proved to, be quite a difficult 
task. The Educational Testing Service (ETS) has these data, but they are 
neither organized by, nor contain information about, the graduate school 
^t which an applicant subsequently enrolls. Thus, to discover the average 
GRE score of students entering, say, the graduate chemistry department at 
the University of Michigan tw) steps were required^rst,^e' contacted 

• The rather large partial correlations for hhtoTfarclntrigu^ probably indicate 4hc 
observation made earlier that scfiools receiving ust) grants had larger first-year enrollments 
before, during^ and after the funding. 
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each graduate department and asked for a list of graduate students who 
had entered in a given year. Second, we sent this list to ETS, which re- 
trieved the test scores for each student and then averaged the scores over 
the department.^ 

Because of the considerable complexity and cost of the retrieval 
process, it was necessary to limit both the number of flelds and the years 
examined. Thus, in March 1974, following a pretest of the retrieval 
instrument, a request was sent to all Roose-Andersen-rated chemistry and 
'history departments asking them to list all students entering their 
departments in the fall of 1964 and the fall of 1973. (The letter was 
addressed to the graduate dean, who in many cases forwarded it to the 
department chairmen. In short, the location of the appropriate data within 
the university varied from institution to institution.) In addition, the 
departments were asked to indicate, where possible, each student's 
baccalaureate institution and (verbal and quantitative) gre score. Most 
institutions were unable to provide gre data, either because of data 
retrieval problems or considerations of confidentiality. Finally, the de- 
partments were asked to indicate the birthdate of each student for whom 
GRE data were not provided in order to facilitate communications with ETS. 

Several reminc'er letters were sent to the nonresponding departments. 
Ultimately, 78 percent of the chemistry departments answered the request 
and provided some or all information, as did 79 percent of the history 
departments. In light of the haphazard recordkeeping of many universities, 
we offered to reimburse each department for any significant clerical time 
involved in retrieving this information. Few responding departments found 
it necessary to draw upon this resource. 

The names of the students whose test scores were not provided by the 
institution were then forwarded to ETS. The information sent back by ets 
for each department ir Jicated the number of students whose test scores 
they were able to retrieve and the mean verbal and quantitative scores of 
the graduate students in that department. At no time did they provide data 
on the test scores of individual students. 

Table 7-11 presents the results of these analyses, indicating both 
**before'' and **after" GRE (verbal and quantitative) aptitude scores in the 
funded field of chemistry and the control field of history for each of the five 

' Lincoln Moses, a member of both the Study Advisory Panel and the Graduate Record 
Examination Board, helped us negotiate this arrangement with bib The Project Officer at Ers 
was Mr. J. Martin Glaubttz. In the processof obtaining these mean gre scores per department, 
rules of stnct confidentiality were observed by both the study and nrs Specifically, no 
individual test scores were used in the analyses reported m this chapter, nor were they 
retained on any data files. The Educational Testing Service was sent a list of names and 
responded with a mean score for each department but did not supply the study with the scores 
of individuals. 
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groups of institutions. Science Development funding seems to have 
significantly affected the quality of the students entering recipient depart- 
ments. In chemistry the change in scores on both the verbal and 
quantitative components of the Gre for students entering the recipient 
departments was much better than that experienced by students entering 
control departments.' The quantitative scores of the former group in- 
creased while it decreased for the latter; scores on the verbal component 
declined for both groups but the decline was much smaller for students 
entering funded schools. These differences and trends were seen in both 
the public and private sector. In history the changes in verbal GRE scores 
for both groups between 1964 and 1973 were similar. However, the 
quantitative scores of students entering history departments at recipient 
schools increased while those of students entering control departments 
decreased. 

In short, there seems to be an effect of Science Development funding on 
the GRE scores of students entering recipient chemistry departments. 
However, the findings from history suggest that the changes on the 
quantitative component of the test may be a result of other institutional 
changes. 

Since GRE data had only been available for a prefunding and a 
postfunding year, it was not possible to use the same multivariate analysis 
techniques as were employed with the other variables. Multiple regres- 
. sions, however, were run and partial correlations computed in a manner 
analagous to that used by the author in his assessment of the effects of the 
NSF College ^cience Improvement program on undergraduates.^ That is, 
two regressions were run in chemistry predicting both the verbal and the 
quantitative GRE score in 1973. In each case the predictors were the 1964 
pretest on that variable and prefunding measures of doctoral production, 
faculty size, publication rate, graduate enrollment and federal science 

• In computing these figures, the diffcrijigsamplc sizes affecting the departmental means were 
taken into account. A weighted sum, computed for each of the groups was divided by the total 
number of students entering departments in that group. The process was not to average the 
separate department means (which were based on widely differing numbers.) 

One potentially complicating factor should be noted. The ability of either the schools or 
ETS to retrieve data on students varied considerably from department to department as a 
function of whether or not the department required this test. Oeariy , the students whoentercd 
departments that did not require the test but who took it anyway constitute a special, biased 
subset of students. As an additional complication, the number of departments requiring the 
CUE changed between 1964 and 1973. 

The key difference— bet ween the 1973 USD and medium control verbal scores for chemistry 
departments— is sutistically significant, assuming that the standard deviation of the scores is 
125 (the figure provided by ets). In point of fact, the actual a for these students probably ii 
smaller in light of their homogeneity, a fact which would increase the strength of the finding. 

* David E. Drew. A Study of the NSF College Science Improvement Program, KCE Research 
Reports. Vol. 6. No. 4, (Washington: American Council on Education, 1971.) 
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funding to the institution. Ihe multiple correlation for the two equations 
were 0.35 (verbal) and 0.38 (quantitative). A partial correlation was 
compuied between each criterion variable and a dummy variable indicating 
whether or not the department received a Science Development grant 
(while controlling for all predictor variables). For the verbal scores the 
partial correlation was 0.3 1 ; .the quantitative measure, 0. 18. These results, 
then, reinforce the zero order fmding about the effect of funding on 
graduate student aptitudes, particularly when contrasted with history 
wherc^ the corresponding partial correlations were -0.28 and 0.08, 
respectively. 

Our other measure of the quality of graduate students was the kind of 
undeipaduate college from which they graduated. As described above. 



TABLE 7-12 Baccatouract^ OHgins ol Students Enttring Sdanct D«velopm«nt snd 
Control Dipartmants, 1964 and 1973 



Number of Students Average Selectivity Score* 





No. 


1964 


1973 


1964 


1973 


AH Institutions 












Chemistry 












USD 


16 


350 


231 


1087.55 


1072.99 


Medium controls 


14 


321 


259 


1085.65 


1063.88 


History 












USD 


14 


366 


219 


1092.78 


1090.42 


Medium controls 


9 


175 


131 


1077.90 


1103.87 


Public Institutions 












Chemistry 










1057.63 


USD 


9 


237 


164 


1081.92 


Medium controls 


11 . 


261 


217' 


1070.11 


1056.81 


History 












USD 


5 


192 


138 


1063.49 


1082.05 


Medium controls 


4 


84 


61 


1056.65 


1057.61 


Private Institutions 












Chf*i!Stry 












USD 


7 


113 


67 


1099.35 


1108.18 


Medium controls 


3 


60 


42 


1153.25 


1100.40 


History 












USD 


9 


174 


81 


1125.09 


1104.68 


Medium controls 


5 


91 


70 


1098.08 


' 1144.19 



• Avcnae selectivity (combined vcfbal plus qtuntiuuive SAT score) cf the students* undergraduate 
coKetet. 
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information about the school of baccalaureate origin for students entering 
chcmisuy and history departnicnts in 1973 and 1964 was collected directly 
from the universities. Each undergraduate college was coded with a 
selectivity measure based on the average aptitude test scores of its entering 
frcshmanv* These undergraduate selectivity scores were then averaged 
over the experimental and control groups. The results from these calcula- 
tions are presented in Table 7-12. Data are presented on both Science 
Development and control institutions for both the pre- and postfunding 
years in chemistry and in history. The informatiol(i is summarized for all 
institutions, public institutions only, and private institutions. 

Inspection of this table reveals no startling differences between the 
behavior of the usD and the control departments in the funded field of 
chemistry, with the possible exception of the private sector. Here the 
selectivity score of students entering funded departments increased 
slighdy at the same time that the score of students entering the control 
departments dropped considerably. Perhaps the most interesting observa- 
tion about this table is how little the baccalaureate selectivity level of 
students entering these departments changed between 1964 and 1973. 

* Alexander W. Astin. Pr f dieting Academic Performancf in College: Selectivity Data for 
2SW^Amencan Colleges. (New York: The Free Press. A Divisioh of the Macmlllan Companyr 
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Impact on the Production 
of Doctorates 



The dawn of the 1970s saw graduate education enter a period of crisis. For 
over a decade, the expansionary thrust of university research had been 
bolstered by increasing amounts of federal support; in the early 1970s, this 
federal support leveled off under the Nixon administration. Concomi- 
tantiy, other traditional sou§pes of revenue became reluctant to support 
graduate education because of a highly publicized surplus of Ph.D.s. One of 
the primary topics considered by the National -Board on Graduate 
Education in its initial statement was this overproduction of Ph.D.s.^ 

A facile criticism sometimes leveled at the Science Development 
program is that it was the right program at the wrong time: i.e., that its main 
effect was to create Ph.D.s who subsequently had difficulty finding 
employment. In light of the salience of this issue to graduate education in 
general, ?Sid to Science Developmentin particular, a special set of analyses 
was devoted to examining the effects of Science Development funding on 
Ph.D. production and to tracing the subsequent employment of Ph.D.s 
from schools that received funding. 

In the mid-1960s, national support for graduate education was at its 
height, and most cxpciis were predicting a snortage of Ph.D.s through the 
next decade. Economist Allan Cartter, however, took a somewhat 
different view, reporting that his analyses and projections indicated that the 
future demand for Ph.Ds would not be likely to exceed the supply. As a 

* Naiional Board on Graduate Education, Graduate Education. Purposes, Problems, and 
Potential, (Washington: National Academy of Sciences, 1972). 

133 



partial explanation of the differences between his conclusions and those of 
other specialists, Cartter noted that ^'educational researchers in govern- 
ment agencies had collected the wrong information for many years and had 
drawn hasty conclusions from imperfect data/'^ Continuing his analysis 
in subsequent articles, Cartter was one of the first, if not the first, to 
predict an oversupply of Ph.D.s beginning in the 1970s. For example, in 
1972, he stated: **We are on a course which would result in one-third too 
many Ph.D.s produced in the latter part of this decade and perhaps 
one-half too many in the 1980s for the types of employment we have 
known in the past.''^ 

In light of the accuracy of his early projections, Cartter's work has 
received a good deal of attention. But he is not without his critics. For 
example, Vaughn and Sjoberg have questioned some of Cartter's assump- 
tions and given a number of reasons for viewing his projections with some 
skepticism. They charge that Cartter ^'ignores fundamental social changes 
already underway within American society, changes that are likely to 
erodethe very basis of his projections.''* Essentially, they argue that basic 
shifts in the nature of the American economy will lead a larger number of 
people to seek higher education than Cartter had assumed. The emerging 
primacy of the service sector in the economy implies a greater reliance on 
advanced education, as does the increase in leisure time and the growing 
demands of women, minorities, and others for advanced education on a 
part- or full-time basis. . 

While national policj;ynakers have been slowly sifting and developing an 
approach to this problem, a number of states have taken direct action. 
Foremost among these is New York, where the State Board of Regents is 
^ invested with considerable authority over both public and private educa- 
tion. A recent report recommended that the number of doctoral-producing 
programs be reduced, citing, among other factors, the overproduction of 
Ph.D.s by the state's higher education institutions.^ A special study has 
been commissioned to review and evaluate, on a field-by-field basis, the 
adequacy of doctoral programs and to make recommendations about 
which should be abolished and which strengthened. 

The growth of graduate education during the 1960s occurred at different 
rates in different sectors of the academic world. For example, as Kidd has 

* Allan M Cartter. "A New Look at the Supply of College Teachers," Educational Record. 
Vol. 46. Summer. 1965. pp. 267-277. 

^ Allan M. Cartter. "Sctentific Manpower of I97(MJ5.** Science, April 9. 1972. p. 243. 

* Ted R. Vaughan and Gideon Sjobcrg. "The Politics of Projection. A Critique of Cartter's 
Analysis,** Science. July 14, 1972. p. 142. ' 

* ** Meeting the Nepds of Di>cloral Education in New York State,** New York Board of 
Regents. Commission of Doctoral Education. Albany. New York. January « 1973. 
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noted, over the decade doctoral production in the top 30 private univer- 
sities dropped from 39 percent to 27 percertt of the total of all doctorates 
4)roduced» whereas in the public universities below the top 30^ it increased 
fro|Ti 9 percent to 24 percent. These percentage changes are set against a 
background in which overall Ph.D. production tripled between .1960 
(10,000) and I%9 (30,000): Among the reasons Kidd notes for the 
differential growth rates of different types of institutions are the steep 
increases in state budgets for support of statejnstitutions in the sixties, 
the great'fer expansion of public universities in"every aspect, and the 
pressure to provide teaching assistants for the rapidly grooving under- 
graduate population at public universities.^ 

Observations such as these about the differential growth rates of public 
and private institutions have led some observers^ to argue that Ph.D. 
output should be limited, essentially to elite institutions. Obviously, this 
reasoning runs counter to the philosophical considerations that led to the 
creation of the Science Development program. 

Clearly, the impact of Science Development funds on Ph.D. production 
and the resulting implications for manpower must be examined carefully 
in a study such as this, but it must not be forgotten that factors other than 
simply manpower considerations should affect the^xlevelopment of higher 
education institutions. For example, Kidd notes that plans for the future 
of graduate education must !ake into account the needs of society and the 
needs of individuals.** Similarly, a National Board on Graduate Education 
statement listed three basic models for studying these issues— manpower 
planning, human capital, free :>tudent choice— and endorsed the third: 

Graduate education is more than investment in human capital and more than a means to train 
people for specific jobs, although it includes both of these. We support the principle of 
free choice for students and believe that it would be a serious error in pubfic policy to close off 
opportunities to potential graduate students on the basis of a centralized manpower plan, or 
because the "investment" may not return the market rate of interest.* 

Thus, our use of Ph.D. production as a key index of quality is justified in 
that this measure is closely related both to current controversies in 
graduate education and to a key criticism of the Science Development 

• Charles V. Kidd. "Shifts in Doctoral Output: History and Outlook." ir/Vm . . Fcbmary 9 
1973 pp. 53»-543. 

Wohn R. Niland, "Allocation of Ph.D. Manpower in the Academic Labor Market," 
Industrial Relations, Vol. 11. No. 2. May. 1972 pp. 141-156. 

Charles V. Kidd, "Doctorate Ouptut— Over Production or Underconsumption," (Speech 
delivered to the American Association for Higher Education, March 13. 1973.) 

• National Board on Graduate Education. Oormrar^ Manpower Forecasts and Policy, No. 2 
(Washington: National Academy of Sciences, 1973). p. 5. ' 
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program. Some observers, of course, consider Ph.D. production to be a 
basic indication of graduate quality. For example. Cartter used a measure 
of doctorate production as the key criterion for inclusion of schools and 
programs to be assessed in his ratings; Roose and Andersen, in their 
replication of his study, followed this lead. In this chapter, then, the rates 
of Ph.D. production in Science Development and in control institutions are. 
compared. This is followed by an analysis of first jobs acquired by recent 
graduates in both the academic world and industry. 



TRENDS IN DOCTORATE PRODUCTION 

This analy^ii^ of trends in Ph,D. production was based on the Doctorate 
Record File produced annually by the National Research Council (NRC). 
Each year since 1958 the NRC has conducted a census of all graduating 
Ph.D.s. Each doctorate recipient completes a four- page questionnaire. The 
samples from these surveys are virtually complete: i.e., the response rates 
are between 99 and 100 percent. While the questionnaires provide a rich 
body of data, the major focus in these analyses was on a simple head count 
of the number of Ph.D.s produced. 

Data from the Doctorate Record File for the 15-year period, 1958 
through 1972. were collected and analyzed. As before, the emphasis was on 
comparing five analytic groups of institutions: USD recipients, DSD 
recipients, high controls, medium controls, and low controls. The findings 
reported below are derived from the examination of the three major science 
fields— physics, chemistry, and mathematics— and the control field of 
history. 

Of course, in comparing Science Development and control institutions 
on a primary variable such as Ph.D. production, one must bear in mind the 
national trends. In Figure 8-1 the number of doctorates produced in each of 
the four fields over the 15-year period is plotted. In each field the total 
number of doctorates produced by all institutions in the sample, whether 
experimental or control, are combined. As the graph shows, doctorate 
production rose steadily through the sixties, peaked in 1970, and has 
tapered off slightly since. This finding holds true for each field and reflects 
the crisis situation in graduate education that touched off the debate of the 
past few years . That is . the tapering-off after the rapid ascent can be seen as 
the system's adjustment to reductions in federal support and to the surplus 
of Ph.D.s. 

Tables 8-1 to 8-4 present the results of the analysis of Ph.D. production 
for each field. The average number of doctorates per institution for each of 
the five groups is given for each year separately for all institutions, public 
only, and private only. 
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Several interesting trends emerge. The most pertinent comparison, of 
course, is between the USD recipients and the medium controls in each 
field. These data arc plotted in Figures 8-2 through 8-5. 

The impact of Science Development funds on Ph.D. production in 
mathematics seems negligible; the only postfunding point af which the 
recipient group greatly exceeded the control group was in 1970. However, 
further inspection revealed striking differences between the trends in the 
public and private domain. The data from the public sector appears to 
indicate that Science Development funds influenced the production of 
doctorates among these universities; in the private sector the opposite 
trend is seen, i.e., the controls outpaced the recipients after funding. 

In physics funding clearly affected Ph.D. production. This can be seen 
both in Figure 8-3 and in the separate breakdowns for the public and private 
sectors. The effect here is more dramatic than in either of the other science 
fields. However, as in those fields, doctorate production by both the USD ^ 
and medium control schools falls well below the number produced by the 
leading institutions. For example, the peak year in doctorate production for 
the USD group in physics was 1971 in which the average uSD department 
produced 15.5 doctorates. In that same year the medium controls produced 
12.8^ doctorates; the high controls, 37.0! 

In chemistry the medium controls exhibited higher rates than the 
recipient institutions before, during, and after the funding. This was true in 
both the private and public sectors. Note also in chemistry that the gap 
between the high controls, which as usual produced the most doctorates, 
and the other two groups is smaller than in the fields of mathematics and 
physics. 

There are some intriguing results in the field of history. Prior to Science 
Development funding the USD group had been producing more doctorates 
per year than the medium controls in history. After funding this trend 
continued and the gap widened in both the public anfl private sectors— a 
finding difficult to interpret. It could represent a spillover effect of Science 
Development funds. Further exploration of this trend in history showed 
that it was sharpest in those institutions that were characterized by 
relatively low levels of federal science support as contr..sted with those 
receiving substantial federal science support. 

Examination of science doctorate production in those schools with 
substantial total federal science support as contrasted with those receiving 
moderate support revealed few differences ir. math and chcmioiry. In 
physics, the effects noted above were seen dramatically in the schools with 
low support, but were not seen in those schools characterized by high 
federal support. Thus, as might be expected. Science Development funds 
had their greatest impact on those schools in which they represented a 
greater share of the budget. 
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FIGURE 8-2 Doctoratt production in sci»nc« dfv«iopm«nt and control Institutions: 
mathtmatios. 
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FIGURE 8-3 Doctorata production in science dtvalopment and control institutions: 
physics. 
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FIGURE 8*5 Doctorite production tn scianc* development ind control institutions: 
hiitorv. 
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As before, these analyses of basic trends were supplemented with a 
multivariate analysis to isolate the unique effects of Science Development 
funds. A number of dimensions representing prefunding departmental 
factors likely to affect subsequent doctoral production were used as the 
independent variables in a series of multiple regression equations: 

1. Total departmental graduate enrollment; 

2. Departmental faculty size; 

3. Departmental publication rate; 

4. Total federal science support for the institution. 

The variables were created in the same manner described in previous 
chapters; in most cases this meant computing the average of the values for 
the years 1959-1%1. 

These independent variables were systematically regressed on doctor- 
ate production for each year from l%5 through 1972. In Table 8-5 the 
results from these regressions are presented as are the partial correlations 
of the dummy variable, indicating whether a department got a grant, with 
the criterion and the partial correlation of total departmental funds with the 
criterion. In Figure 8-6 the mean residual scores (actual doctorate 
production minus predicted doctorate production) for the two key 
groups are plotted for the period 1%5-I972. Inspection of these results 
indicates that/ basically, these multivariate analyses substantiate the 
trends observed in the zero-order profiles. 



COMPARISON OF Ph,D.s' FIRST JOBS 

The focus of the Science Development progrann and of this assessment was 
on quality in graduate education. Though rate of Ph.D. production is an 
important index, it is basically quantitative. To supplement the above 
analyses with some that addressed the issue of quality more directly, we 
examined the nature of the first jobs taken by Ph.D.s from recipient and 
control institutions. 

The unemployment of Ph.D.s has received considerable attention in the 
media, of course. But for virtually all of the years covered by this study, the 
percentage of unemployed Ph.D.s was so low that it was imnnssible to 
examine these trends meaningfully. In addition, as Niland has commented: 

For most of the lower skills, the rate of unemployment serves as a reasonably accurate 
barometer of labor market conditions. However, for the more highly skilled the slate of the 
market may be only weakly reflected in the employment rate. High level manpower tends to 
by mobile and thus has the ability, mainly by working in lesser capacities, to disguise or at 
least delay detection of a liwsening market.'^ 

Niland. op. cit.. p. 14!. 
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Figure 8-6 Rosidual analysis of doctorate production: (a) mathematics, (b) physics, 
(c) chemistry, (d) history. 
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Thus, we focused not on unemployment rates but on a comparison of 
the degree to which graduates of recipient and control departmeiUb were 
^*wor4cing in lesser capacities." Those Ph.D.s who were employed after 
receiving their degree could have been working in either academic or non- 
academic jobs. Some of the characteristics associated with both types of 
employmentM||p(xamined below. 

The analyses of post-Ph.D. academic employment were conducted 
'using national data from the Doctorate Record File of the NRC and the 
National Register compiled by NSF. The latter, a biannual survey con- 
ducted through the scientific professional societies, gathers information on 
all scientists and engineers currently working in the United States; one 
subgroup is scientists who have received their Ph.D. in a given year. The 
overall response rate is about 70 percent, though it varies from field to fielcf! 
Although the survey ask« about current employment only, we have 
assumed that this represented the first job for those who had received their 
Ph.D. within the previous two years. During the 15-year period covered by 
this study. National Register surveys including a question on current 
employment were conducted in 1964 and 1966. Using these surveys, we 
gathered data on the postdoctorate employment of graduates from the 
1%2, 1%3, 1964, 1%5, and 1966 cohorts. For each subsequent year we 
drew upon the Doctorate Record File. Beginning in 1967 with the 
unfortunate omission of 1969 and 1970, the Doctorate Record File survey 
instrument asked about the institution at which the Ph. D. planned to work. 

In summary, data on postdoctorate academic employment were 
gathered for the years 1962-1968 and 1971-1972 by combined use of the 
National Register and the Doctorate Record File. The latter provided more 
complete survey information than the former. 

Table 8-6 presents data on the postgraduate academic employment of 
Ph.D.s from recipient and control institutions for physics, chemistry, and 
mathematics.** Each entry is the average Cartter faculty quality rating of 
the departments to which graduates from that group of institutions in that 
year went to teach. To illustrate, in 1968, the typical graduate of a USD 
mathematics department took his first job at an institution where the 
Cartter rating of the math department was 1.35. Unfortunately, if one 
assumes that Ph.D.s whose graduate training was strongly affected by 
Science Development funding graduated in 1968 at the earliest, the only 
clear measure of postfunding employment are the data on 1971 and 1972. 

The data for the USD and medium control groups are plotted in Figures 
8-7. 8-8. and 8-9. 

Some of the entries in Table 8-6 had to be based on relatively small 

'* The National Register docs not provide information about Ph.D s from the control field of 
history. 
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FIGURE 8-7 Average Cartter rating of academic departments at which graduates of science 
development and contro) institutions took their first job: mathematics. 
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FIGURE 8.6 Average Cartter rating of academic departments at which graduates of science 
development and control institutions took their first job: physics. 
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FIGURE 8 9 Average Cartter riting of academic departments at which graduates of science 
development and control mstitutions took their first job: chemistry. 

samples. As an aid to the r^eader the sample sizes on which all Table 8''6 
entries are based are given in Table 8-7. 

As one might expect, Ph.D.s from ail levels of institutions are going to 
less highly ranked institutions these days than in the past as a result of the 
tight job market in academia. Virtually all of the trends reported in this table 
are down. This is the so-called trickle effect that Niland and others have 
discussed. 

Table 8-6 reveals that while all graduates were going to poorer jobs in the 
later years, this was relatively less true of Science Development graduates 
than of control graduates from private institutions in mathematics. Other- 
wise the USDS and medium controls were landing comparable jobs in the 
most recent years. 

New doctorate recipients often seek jobs outside the academic sphere; 
the nonacademic, industrial sector is a major employer of these scientists. 

A subgroup (approximately one third) from each graduating cohort took jobs at under- 
graduate colleges. i,e . in departments which were not rated by ( artter In the data reported 
above these departments were assigned a Cartter rating of zero 

In addition, aspetialrcanalysis of the ddtawasconducted in which this subgroup of Ph D.s 
was excluded When only Fh D s who look jobs at Cartter-ratcd departments were studied, 
the result did not differ nw^rkedly from those reported above. Hssentially, in all three science 
fields. Ph D s from the USD and the medium control groups were found to be taking jobs 
in comparably rated departments m 1971 and 1972 
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FIGURE 8<10 Avaragt salarits r«cttved by graduates of scionce davetopnxint and control 
institutions in their first nonacademic job: physics. 
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Thus, we also analyzed the first jobs of those Ph. D.s from experimental and 
control institutions who did not enter college and university jobs. This 
analysis drew exclusively upon the National Register, which, in every 
survey conducted during the time frame under consideration, had asked the 
respondent about his current salary. Salary was the one indicator of the 
status of an industrial job that was available for this research. 

Table 8-8 shows the mean salaries of Ph.D.s from each group of 
institutions for the years 1958 through 1970 for physics and chemistry. In 
Figures 8-10 and 8- 1 1 , the means for the USDand medium control groups are 
plotted for each of the two fields. In Table 8-9 the corresponding sample 
sizes are given. (The samples for mathematics and for the dsds in physics 
were so sparse that the data had to be omitted.) 

Some obvious general conclusions can be drawn. Salaries rose at afoirly 
steady rate over the years, reflecting inflation. The salaries of graduates 
from Science Development institutions and from control institutions 
climbed at about the same rate. In general. Science Development funds did 
not seem to have a major effect on this trend, nor did the graduates from the 
recipient institutions suffer dramatically in comparison with the controls. 
In short, the salaries achieved by Ph.D.s from the two groups tended to be 
about the same at all time points.*^ 

A close inspection of Tabic 8-8 reveals very few differences in salary levei.^» even between 
the high and the low controls Thus, although tracing salary data seemed a valuable exercise 
(particularly since it was the sole index available of the status of an industnal job), this 
parameter cannot be considered a measure of graduate depailmcnt quality . This» in itsclf» is an 
intriguing finding. 
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Summary 



This technical report has presented in detail the findings from an evaluative 
study of the National Science Foundation's Science Development pro- 
gram. The study was requested by the NSF Evaluation Unit in IS)7I, the 
same year in which the funding program ended. 

The Science Development program (described in detail in Chapter 2) 
was a pioneering experiment in institutional funding initiated in 1965 with 
the goal of dramatically increasing the number of **centers of excellence'' 
in graduate science education. . In the early 1960s criticism had been di- 
rected against the concentration of federal funds and national science talent 
in a few universities, disproportionately clustered in the Northeast. The 
major objective of Science Development was to stimulate high-level 
science activity at second-tier institutions dispersed geographically 
throughout the nation. 

There were three subprograms under Science Development. By far the 
largest and most important was University Science Development (USD), 
under which funds totaling $177 million were given to 31 universities that 
seemed to have potential for developing superior science capabilities. 
Under USD, several science departments—typically four or five — in each 
university were funded over a period of five to seven years. The second 
subprogram. Departmental Science Development (DSD), awarded73 grants 
to single departments considered to have the potential for high quality but 
located in universities that generally were too weak to qualify for the larger 
USD subprogram. Finally, II Special Science Development (SSD) grants 
were given to universities that fell between the two poles represented by 
USD and DSD- 
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The basic purpose of this evaluation study was to examine the impact of 
Science Development funding on the recipient departments and institu- 
tions. A second objective was to assess the relatixe efficacy of the 
subprograms. In addition to these fundamental issues, the study was 
concerned with a number of related questions: Were nonfunded depart- 
ments in funded universities affected? What happened in the funded 
universities when these massive government grantsen3ea?The methodol- 
ogy was developed to address these substantive concerns, as well as to 
satisfy the technical demands of effective assessment. 



METHODOLOGY 

Answering the basic questions required tackling the thorny issue of the 
meaning of ''quality'' science education in American graduate schools. An 
early decision was made to use multiple indicators or :riteria of quality . We 
hope that this research has contributed toward a definition of quality 
graduate education in terms that can be operationalized for scientific 
research. 

The methodology combined two approaches: (1) case studies, carried 
out by site visits at selected institutions, and (2) multivariate analysis of 
quantitative data on institutions. 

The site visits were conducted at 16 recipient institutions and five 
nonfunded (control) institutions, which were studied for purposes of 
comparison. Of the recipient institutions, nine had been awarded USD 
grants, six had received one or more DSD grants, and one had been awarded 
an SSD grant. For the smaller DSD recipients, the site- visit teams included a 
senior member of the project staff (typically the project director) and a 
scientist who had been on the original nsf team that reviewed the proposal 
submitted by the university. For the USD and ssD recipients, the teams 
were larger; in addition to a senior member of the project staff, they 
included one or two scientists who had visited the university previously, 
skilled and experienced academic administrators, and researchers 
specializing in higher education asj»essment. Where possible, members of 
the Study Advisory Panel and the National Board on Graduate Education 
joined the team. 

A wide range of institutions was visited: public and private; funded and 
nonfunded; those where the grants were thought to be successful and those 
where they were thought to have failed. However, in light of the nsf 
emphasis on regional dispersion of the funds, we deliberately attempted to 
visit schools all over the country, fable 3-1 summarizes the distribution 
of site-visit institutions with respect to control (public or private), region, 
and type of grant. 

At each institution, members of the site team met with the chief 
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administrator, the institutional representative for the Science Develop- 
ment grant, provosts, Ueans, and other high officials. In addition, they 
conducted in-depth visits to both funded and nonfunded departments, 
inte^-viewing vlepartment chairmen, faculty members, and^ graduate stu- 
dents and examining equipi^'^nt and facilities acquired under Scjence 
Development support. 

For the quantitative analyses, both longituUinal data and complex 
statistical techniques were required to isolate the unique effects of Science 
'Development. Wherever possible, the information gathered for this study 
covered the 15 years froni 1958 through 1972. Thus, by going back for 
several years prior to the launching of Science Development (1%5), we 
were able to identify trends already under way at the institutions. 

To assess the impact Of Science Development, it was necessary to 
compare trends in fundeu institutions with trends in nonfunded (control) 
institutions. So that these comparisons would be as thorough and as 
meaningful as possible, we selected for the control group not just a few 
institutions but every major doctorate-producing institution in the nation. 
I'hus, the sample comprised all universities rated by Roose and Andersen 
m their l%9 evaluation of graduate education in the United States. This 
procedure enabled us to compare Science Development mstitutions with 
nonfunded institutions of roughly similar initial quality (the 'medium 
controls"), with universities considered already outstanding in science (the 
**high controls"), and with institutions whose science capabilities were 
considered generally unpromising (the "low controls"). 

As indicated above, multiple criteria were used to operationalize the 
concept of science quality. These criteria were faculty size and publication 
rates, graduate student enrollment size, test scores anu baccalaureate 
origins of graduate students, Ph.D. production, and characteristics of 
institutions at which Ph.D.s took their first jobs. 



SITE VISITS 

The case studies that emerged from the site visits yielded a number of 
general observations about the effects — both good and bad — of Science 
Development. 

Clearly, Science Development funds were responsible for many desira- 
ble changes; some universities made a quantum leap forward with the aid of 
these funds. Concrete evidence of the program's positive impact include 
brilliant new faculty members acquired, graduate enrollments increased, 
buildmgh constructed, a telescope built, exciting interdisciplinary institutes 
formed, etc. Basically, Science Development funds served as a cata- 
lyst, allowing the recipient universities to accelerate the development 
of their science capabilities. While it would not be accurate to say that the 




program created ''centers of excellence'* in the sense that all the funded 
institutions are now the academic peers of Berkeley and Cal Tech, the 
improvement was, in most cases, notable. 

The program's most obvious negative effect was tied to the erratic 
pattern of overall federal support for graduate education. In most schools, 
the support from Science Development was ending at the same time that 
the widely publicized financial crunch in higher education hit. Many 
institutions had to struggle in order to fulfill their commitment to maintain 
the improvements initiated under Science Development. All too often, 
other departments in the institution suffered as a result of this emphasis on 
the sciences. In short. Science Development funds had the effect of 
distorting the overall balance at some institutions. 

Perhaps the key factor that distinguished successful grants from those 
that were less so w^s the strength of an institution's central administration, 
especially its president orchancellor. This person's continuity in office was 
particularly important. Grants tended to be regarded as most successful at 
universities that had a strong and dynamic leader who was in office before, 
during, and after the grant. An eanally important factor — and one closely 
associated with a strong central administration — was the existence of a 
detailed development plan. Such plans often served as the foundation for a 
Science Development proposal that then was likely to contribute to 
balanced overall growth. 

In fact, one of the secondary benefits of Science Development was that it 
forced those universities that had previously failed to plan for science to 
develop such plans. Thus, even those institutions whose requests for 
funding were rejected by the Foundation may have reaped some advantage 
from the organized thought required to develop a proposal. 

Science Development funds had a number of indirect effects on 
nonfunded departments at recipient institutions. For instance, the de- 
velopment o^support facilities like a computer center may have benefited a 
spectrum of departments. As another example, a particular department at 
an institution that received a USD grant may have been judged too weak to 
merit funding; in some cases, the institution itself then made special efforts 
to develop this weak department at the same time that USD funding was 
building up the other science areas. 

The site-visit teams heard numerous testimonials to the flexibility of 
those NSF officials who had administered the program and worked with the 
institutions. A retrospective comparison of funded vs. nonfunded institu- 
tions, and of funded vs. nonfunded departments in the same institution, 
attested to the validity of NSr's judgment in selecting or rejecting 
applicants. In retrospect, however, it seems clear that the Foundation 
should have required more in the way of evaluation and documentation 
from the institutions as a condition of receiving these massive grants. 
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QUANTITATIVE ANALYSIS 

The use of quantitative analyses permitted us to pinr 'int the unique effects 
of Science .Development funding on the criteria of excellence listed above. 
All analyses were repeated in each of the three primary science fields 
funded by the program (mathematics, physics, and chemistry) and in a 
control field, history. 

Faculty Size 

Since a large segment of a typical Science Development award was 
earmarked for **personner' and since most scientists believe that the way 
to develop a department is to hire excellent people, we first questioned 
whether changes in departmental faculty-sizes resulted from the funding. 
The results of this analysis were clear-cut and dramatic. There is no 
question that the NSF funds enabled the recipient departments to increase 
their faculties. This growth was evident in the three science fields but not in 
history. For physics and chemistry, the growth in faculty size was limited 
to the public sector; in the private sector the recipient institutions had 
larger faculties than the controls prior to funding, a difference that persisted 
through 1972. 

A central issue surrounding Science Development was whether these 
increases in the faculty sizes of funded departments were made at the 
expense of nonfunded departments, particularly those in the leading 
institutions. Some critics of the program charged that it represented no net 
gain for ttie nation but merely enabled the recipient institutions to **steal" 
outstandir^gscientistsfromthe*'best" research universities. This assertion 
was subjected to an empirical test in a special analysis of faculty mobility in 
the field of physics (see Chapter 5). The analysis revealed that grants 
allowed the recipient institutions to maintain the growth rate established 
prior to funding, whereas the growth rate of the medium controls slowed 
down considerably during the funding period. The sources of senior facuhy 
for the physics recipients under funding did not differ markedly from that 
before funding. Moreover, it is clear that any defections by senior faculty 
from leading institutions to the USD recipients were trivial before, during, 
and after funding. 

■> ♦ 

Faculty Productivity 

Next, we looked at scholarly productivity, as measured by rate of 
publication in key journals. To select the leading journals in physics, 
chemistry, and mathematics, we ranked-ordered all journals in each field 
on the basis of an impact factor: i.e. , the ratio of citations to a given journal 
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(in other journals) divided SyTRe number of source articles in that journal. 
(A slightly different, but analogous, procedure was necessary for the con- 
trol field of history.) The journal at the top of the list, then, was the one in 
which an article would have the greatest impact in that other scientists 
would cite it frequently in their work. During the ranking process, various 
technical adjustn>:jnts were made for such factors as the representation of 
American authors in English-language journals published in other coun> 
tries. The list for each field was limited to about 20 leading journals, but 
these journals accounted for a majority of the citations in that field. In phys- 
ics, for example, ihe list of 20 represented over 75 percent of all citations. 

Next, the rale of publication in these journals by faculty in funded and in 
control institutions was plotted between 1958 and 1972. In addition, to 
isolate the unique effects of Science Development, we cairied out a 
multivanate analysis, in which residual scores were derived. 

National Science Foundation funding had a positive impact on de- 
partmental productivity in each of the three science fields. In chemistry this 
effect was revealed only through the multivariate analysis, but in math and 
physics it was clear from the zero-order analysis. The increase in 
departmental publication rates at funded institutions was greatest in 
mathematics, definite in physics, and slight in chemistry. The trend in 
mathematics was found in both the public and pcjvate sectors. 

The Science Development program had less impact on per person 
productivity. Although the data for mathematics seemed to indicate an 
increase in the publication rates of individual faculty members, the residual 
analysis raises a question as to whether this increase was attributable to 
Science Development funding or was an artifact of other factors. The 
multivariate analysis revealed a possible impact on the publication rates of 
individual chemists. No such effects were found for either physics or the 
control field of history. 

In short, recipient science departments clearly improved their publica- 
tion rates with the aid of Science Development funding. But this improve- 
ment was a departmental change, probably resulting in large part simply 
from the increase in faculty size. The effects on per person productivity 
were minimal. 

Graduate Students: Enrollments and Quality / 



Next, the impact of Science Development funding on enrollments, a 
quantitative measure, and quality, defined in terms of the ORE test scores 
and baccalaureate origins of entering graduate students, was assessed. 
The receipt of a Science Development grant had very little effect either 
on total graduate enrollments or on first-year, full-time graduate enroll- 
ments. The one exception was found among public recipient institutions, 
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where graduate enrollments in mathematics increased* apparently as a re- 
sult of the funding. 

A special retrieval effort was conducted to gather information on the 
GRE scores of students entering a funded department, chemistry, and 
the control department of history in a prefunding and a postfunding 
year. Comparison of the changes revealed that the former group improved 
their relative standing signiHcantly during this time period. In short, there 
seemed to be an effect of Science Development funding upon the GRE 
scores of students entering recipient chemistry departments. However* 
findings from the control field of history suggested that the changes on 
the quantitative component of the test might be the result of factors other 
than NSF funding. 

As another measure of the quality of graduate students, we looked at 
the baccalaureate origins <^ students entering funded and control graduate 
departments. Specifically, \ye wanted to see if the graduate students at- 
tracted to the recipient dep/irtments after funding tended to come from 
more highly selective undergraduate institutions. No such trend was found. 

Doctorate Production 

The crisis facing graduate education today derives in part from the highly 
publicized surplus of Ph. D.s in some fields, a surplus indicated by the rela- 
tively high unemployment rates of this select group. Some critics have 
charged that the Science Development program contributed to this problem 
by accelerating the production of Pfi/iD.s. A special set of analyses was 
devoted to examining the effects of Science Development funding on Ph.D. 
production and to tracing the subsequent employment of Ph. D.s from re- 
cipient and control institutions. ^ . 

Data from the Doctorate Record File of the National Research Council 
allowed us to plot Ph . D. production in each of the three science fields and in 
history for the period 1958-1972. In mathematics, the data indicated thai 
Science Development funds increased the production of doctorates in the 
public sector, butjiot in tha private xse^^tor. National Science Foundation 
funding had its most dramatic effect on Ph.D. production in physics, where 
it was apparent in both the public and the private sectors. In chemistry, no 
relationship was found between funding and the production of Ph. D.s. It 
should be noted that even in mathematics and physics, the number of 
Ph. D.s from both the USD and the medium control schools fell well below 
the number from the leading institutions. For example, in 1971 — the peak 
year for the USD group in physics — the average recipient department 
produced 15.5 doctorates* the medium controls produced 12.8 doctorates, 
but the high controls produced 37.0! 

The rate of Ph.D. production, though important, is a quantitative 
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measure. To get at the issue of quality more directly, the nature of the first 
Ijobs taken by Ph.D.sfrom recipient and control institutions were reviewed; 
separate analyses for those entering academia and those taking non- 
academic jobs were performed. 

For the former, we examined the ICartterJ faculty quality rating given to 
the department at which graduates of funded and of control institutions 
took their first job. As one might expect, in view of the tight job market in 
academia, Ph.D.s from all groups of institutions are going into more poorly 
ranked institutions these days than in the past. But this was less true for 
Science Development Ph.D.s than for control Ph^D.sfrom private institu- 
tions in the field of mathematics. Otherwise, Ph.D.s from funded and from 
control institutions fared about the same in the postfunding years. 

Salary was the one indicator of the status of a nonacademic job that was 
available for this research. Analysis revealed that the starting salaries of 
graduates from Science Development schools and from control schools 
rose at about the same rate. 



CONCLUSIONS 

Most evaluation studies of federal funding programs are hard pressed to 
find any discernible impact. It is clear at the conclusion of this study that 
the NSF Science Development program was a powerful enough force in 
graduate education that several years after its demise, its impact can be 
seen in a variety of ways. On the face of it, some of these impacts were 
positive, some negative. The goal of this study has been to determine 
objectively whether or not effects occurred. The interpretation of their 
value is a philosophical decision that we leave to the reader. 
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Appendix A 

Composition of Samples 



MATHEMATICS 

USD Recipients 

Arizona 
Brooklyn 

Polytechnic Institute 
Camegie-Meiion 
Colorado 
Florida 

Louisiana State 
Maryland 
Michigan State 
Notre Dame 
Oregon 
Rice 
Rutgers 
Tulune 

use 

Virginia 

DSD (and Special) Reci- 
pients 

Claremont 

Denver 

Kentucky 

New Mexico State 

RPI 
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High Controls 

Brandeis 
Brown 

California. Berkeley 
California Institute of 

Technology 
Chicago 
Columbia 
Cornell 
Harvard 
Illinois 
Michigan 
Minnesota 
MIT 

Pnnceton 

Stanford 

UCLA 

Wisconsin 

Yale 

Medium Controls 

Illinois Institute of 

Technology 
Iowa State (Ame?>) 
Johns Hopkins 
Kansas 
Lehigh 
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Missouri 
Northwestern 
Ohio State 
Oklahoma 
Oregon State 
Penn State 
Pennsylvania 
Syracuse 
Wayne State 
Yeshiva 

Low Controls 

Alabama 

American 

Boston University 

Catholic 

Cincinnati 

George Peabody 

George Washington 

Nebraska 

Oklahoma State 

St. Louis 

SUNY. Buffalo 

Tennessee 

Texas A&M 

Utah 

VPI 

Wa^h ngton State 



PHYSICS 

USD Receipients 

Arizona 

Brooklyn Polytechnic Institute 

Camcgic-Mellon 

Case Western Reserve 

Colorado ^ 

Duke 

Florida 

Florida State 

Indiana 

Louisiana State 

Maryland ' 

Michigan State 

New York University 

North Carolina 

Notre Dame 

Oregon 

Pittsburgh 

Purdue 

Rutgers 

Texas 

use 

Vanderbilt 
Virginia 

Washington (Seattle) 
Washington (St Louis) 



DSD (and Special) Rect* 
pients 

Brandeis 

Delaware 

Nebraska 

Ohio University 

Stevens Institute 

Tennessee 

Utah 

Washington State 
Yeshiva 



High Controls 

California* Berkeley 
California Institute of 

Technology 
Chicago 
Columbia 
Cornell 



Harvard 
Illinois 
Michigan 
MIT 

Pennsylvania 

Princeton 

Stanford 

Wisconsin 

Yale 

Modium Controls 

Boston University 

Brown 

Catholic 

Illmor Institute of 

Technology 
Iowa State (Ames) 
Johns Hopkins 
Kansas 
Lehigh 
Minnesota 
Northwestern 
Ohio State 
Oregon State 
Penn. State 
RPI 

Syracuse 
fexas A&M 
Tufts 
UCLA 
Wayne State 

low Controls 

Alabama 

Bryn Mawr 

Cincinnati 

Connecticut 

Emory 

Fordham 

Georgetown 

George Washington 

Georgia Tech 

Kansas State 

Kentucky 

Missouri 

New Mexico 

Oklahoma 

St. Louis 

SUNY, Buffalo 

femple 
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VPI 

West Virginia 
CHEMISTRY 

USD RecipienU 

Arizona 

Brooklyn Polytechnic Institute 

Carnegie-Mellon 

Case Western Reserve 

Colorado 

Duke 

Florida 

Rorida State 

Indiana 

Louisiana State 
Maryland 
Michigan State 
North Carolina 
Notre Dame 
Oregon 
Pittsburgh 
Rochester 
Texas 

use 

Vanderbilt 

Virginia J 
Washington (St. Louis) 

DSD (and Special) Reci- 
pients 

Brandeis 
Bryn Mawr 
Emory 
Hawaii 
Nebraska 
Oregon State 
RPI 

South Carolina 
I'exas A&M 
Utah 

Washington State 
Wayne State 

High Controls 

California. Berkeley 
California Institute of 
Technology 



ERIC 



Chicago 

Columbia 

Cornell 

Harvard 

Illinois 

Minnesota 

MIT 

Northwestern 

Princeton 

Stanford 

UCLA 

Wisconsin 

Yale 

Medium Controls 

Brown 

California, Davis 
Cincinnati 
Delaware 
\ Georgia Tech 

Illinois Institute of 

Technology 
Iowa State (Ames) 
Johns Hopkins 
Kansas 
Kansas State 
Massachusetts 
Michigan 
Ohio State 
Penn State 
Pennsylvania 
SUNY. Buffalo 
Tennessee 

Low Controls 

Alabama 
Arkansas 

K Boston University 
Catholic 
Clark 
Connecticut 
Fordham 
Georgetown 
George Washington 
Houston 
Kentucky 
fxhigh 

Loyola (Chicago) 
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Missouri 

New Mexico 

Oklahoma 

Oklahoma State 

St. Louis 

Syracuse 

Temple 

Tufts 

VPI 

West Virginia 
Wyoming 



HISTORY 

USD Recipients 

Case Western Kvserve 

Colorado 

Duke 

Florida 

Indiana 

Iowa (Iowa City) 
Louisiana State 
Maryland 
Michigan State 
New York University 
North Carolina 
Notre Dame 
Oregon 
Pittsburgh 
Rice 

Rochester 
Rutgers 
Texas 
I u lane 

use 

Vanderbilt 
Viigmia 

Washmgton (Seattle) 
Washington (Si Loi 

High Controls 

California, Berkeley 

Chicago 

Columbia 

Cornell 

Harvard 

Johns Hopkins 
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Michigan 

Northwestern 

Pennsylvania 

Princeton 

Stanford 

UCLA 

Wisconsin 

Yale 

Medium Controls 

Brandeis 

Brown 

Claremont 

Clark 

Emory 

Gerogetown 

George Washington 

Illinois 

Kansas 

Kentucky 

Minnesota 

Missouri 

Nebraska 

Ohio State 

Oklahoma 

Penn State 

Syracuse 

Wayne State 

Low Controls 

Alabama 
American 
Arkansas 
V Boston University 
Catholic 
Cincinnati 
Fordham 
George Pcabody 
Lehigh 

Loyola (Chicago) 
New 'Mexico 
North Dakota 
St. Johns (N.Y.) 
St. Louis 
SUNY, Buffalo 
Utah 

Washington State 
West Virginia 



SAMPLE GROUPS BASED ON THE 
COMBINED SCIENCE RATING 



USD Racipi«nts 

Arizona 

Brooklyn Polytechnic Institute 

Camegie-Mellon 

Case Western Reserve 

Colorado 

Duke 

Florida 

Florida State 

Indiana 

Iowa (Iowa City) 
Louisiana Stae 
Maryland 
Michigan State 
New York University 
North Carolina State 
North Carolina 
Notre Dame 
Oregon 
Pittsburgh 
Purdue 
Rice 

Rochester 
Rutgers 
Texas 
I'ulane 

use 

Vanderbilt 
Virginia 

Washington (Seattle) 
Washington (St. Louis) 



High Controls 

California. Berkeley 

California Institute of Technology 



Chicago 

Columbia 

Cornell 

Harvard 

Illinois 

Michigan 

Minnesota 

MIT 

Pnnceton 

Sr;»nford 

UCLA 

Wisconsin 

Yale 



Medium Controls 

Brandeis 
Brown 

Illinois Institute of Technology 
Iowa State (Ames) 
Johns Hopkins 
Kansas 
Nebraska 
Northwestern 
Ohio State 
Oregon State 
.Pcnn State 
' Pennsylvania 
RPl 

Syracuse 
Tennessee ' 
Utah 

Washington Slate 
Wayne State 



Low Controls 

Alabama 

American 

Arkansas 

Boston University 

Bryn Mawr / 

California, Davis ' 

Catholic 

Cincinnati 

Clark 

Connecticut 
Delaware 
Fordham 
Emory 

George Peabody 
Georgetown 
George Washington 
Georgia Tech 
Houston 
Kansas State 
Kentucky 
Lehigh 

Loyola (Chicago) 
Massachusetts 
Missoun 
New Mexico 
Oklahoma 
Oklahoma State 
St. Louis 
SUNY, Buffalo 
Temple 
Texas A&M 
Tufts 
VPI 

West Virginia 

Wyoming 

Yeshiva 
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Appendix B 

Selection of Key Journals 



To develop the criteria for selecting journals, we drew on an isi analysis of 
their 1%9 data, in which the leading 1,000 journals in science had been 
rank-ordered both by number of publications and by number of citations. A 
third ranking, which took into consideration an ''impact factor,** most 
closely met the needs of this study. The impact factor was the ratio of 
citations to source articles; thus, journals ranking high on this list were 
those whose articles were most frequently cited, indicating that the impact 
of an article in that journal would be maximal. Between acquisition of the 
ISI list and the final selection of journals for this study, however, we had to 
make a number of more fine-tuned decisions. 

The analyses reported in Chapter 6 cover four fields: physics, mathema- 
tics, chemistry, and history. Since the control field of history was not 
covered by the isi data bank, journals in that field were selected by special 
methods parallel to those used in the otfier three fields. 

First, all journals in a given field were selected from the list of 1,000: In 
mathematics there were 33; in physics, 66; and in chemistry » 69. Then the 
journals for a given field — say, mathematics— were rank-ordered by im- 
pact factor. Each journal was examined carefully for possible idiosyn- 
crasies or problems. 

The m^jor technical problem in developing the list for each field was the 
presence of foreign journals. For a variety of reasons, we decided to 
exclude journals that were not published in the English language. Those 
foreign' journals in which the work of American authors appear to any 
substantial extent tend to be English-language periodicals published in one 
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of three countries. Canada. Great Britain, and Holland. Journals published 
in other countries (whether in the Hnglish language or not) were also 
eliminated. 

The impact of an eligible foreign journal was measured in terms of the 
American influence reflected in that journal. That is. a count was made of 
the proportion of articles authored by American scientists as opposed to, 
say. British or Canadian scientists. This proportion was then used to adjust 
the other figures in the selection process such as the impact factor. To 
illustrate, the British journal, A JvY//itt'.s in Fhy.sUs. has a total impact factor 
of 3.837 but the proportion of American scientists publishing in that journal 
is 13 percent. Thus, the adjusted impact factor for that journal was 0.512. 
Similiarly. the counts of the number of citations to that journal and of the 
number of source articles in it were adjusted by this proportion. 

As a result of this method, the 20 physics journals selected repre- 
sent, in sequential order, the 20 journal outlets in which scholarly articles 
wntten by American physicists have the greatest impact. 

An additional problem existed in that neither the academic world nor the 
world of scholarly publication is divided into neatly defined fields. Many 
journals deal w ith more than one field . Our options here were ( I ) to drop the 
multifield journal entirely . (2 ) to include it in both fields, assigning the same 
impact factor to each, or (3) to include it in both fields, assigning a different 
impact factor in each. 

Were the first option adopted, much significant information would be 
lost. There appears to be no valid basis on which to make the distinctions 
necessary for the last option. Therefore, we decided to include multifield 
journals, assigning an equal impact factor for both fields involved. 

Some additional criteria were applied: Abstracts and bibliographic jour- 
nals were excluded, as were journals that began publication after De- 
cember 3 1 . I%2 . though the latter group was included in the counts of total 
citations and total source articles. The few journals that do not list the 
corporate addresses of authors were, of necessity, excluded. 

Tables B-l. B-2 and B-3 list, in order of their impact, the journals 
.selected for the fields of mathematics, physics, and chemistry, respec- 
tively. Each table includes the journal title, the impact factor, the citation 
count (I.e.. all l%9citations to thejournalforarticles published in l%7and 
1968). the number of source articles (i.e.. all source articles published in the 
journal in l%7 and 1968). and the total citation count for that journal in 
l%9* (I.e., all Citations to the journal in 1%9 for articles published in any 
year). As indicated above, figures for foreign journals have been adjusted 
to leflect the representation of American authors. In addition, the total 

' In fact, isi computed these ngiires by multiplying the *.ounts for Ihe List quarter of l%y by 
four 

173 

ERIC 



5i £ 52 
(- Z O < 



S S « 3 

— v-i rsj r-i 



r- Tt — O 



— — o 



o f*% ^ 

- — C7V 4 

v% %c — <o — 
— ' f^' 



On rs| ^ r*% 
O 00 
rs| rsi m 



< 



o ^ 
C/5 w 



rsi rsi >o r-i 0> 



(Tk 



s g ? s 

Sri y 

2 G 5 < 



■^00»/^r^f*%O\»O»o— ^*rk — — vS — 



rsi o — 



^ o -v; 
•^i r? 



vC ^ — ri */^ 

r<^ — ' r-i m 00 — r^'O 

O O 0> *r> v'. r^, t^i 

r-J — — — — — 



"3 



^ ^ 



c 5; 



~ C r» ^: 

•t: i;, ft; i;. 



'Z c > 

• ^ <r, 



^ ^ ^ k> £ 

3 5 5 I ,t 

ftj V 



a, ^ Ji, Uj 



174 



prj « so^ so *^ <H 



oooo^^?\»o — — — 



(N f*% ^ «^ 



— NO 
^ « 



Sr-* On — &s r-* On 
•Tk OS »/% f*^ «r> 



xr^&r^^OsOON^ 



— m — (N 



8 S ^ « :£ S S S S 
^ :q « Ss ; t- 7t 

rsi « ^ — CM 



3 5 



OON»O^OOvr»SN^«^'-" 



ir^o\ — — (N — 5252 



3! 



*3 ^ 
5 



'I 



5 5 s 



E I 



1 



11 
^1 



5 ^ t 

.a -a -3 
5 5 g 



S 3 



•Si ^ 
§1 



5; o 



"5 c . 

<y 
T 

Q Q ^ 



5 .a 

"a B 

•Si ? 

1 1 



2 ^ .2 

-So 

1;^ ? b 



il 



5 

t g 5: g» 



S I 2 
i a ^ 



o w 



I- 

O 

ft, ■ 



- 5 g 
^ ^ 



7 

a c p 

IP 



175 



numbers of citations to, and source articles from, the 20 selected journals 
combined, all journals in the fielJl, and the proportion of the latter consti- 
tuted by the former are summarized at the bottom of each table. ^ 

'Because history was not covered by isi, it posed a special problem. An' 
examination of the literature in history and consultation with historians 
revealed that the American Historical Association's publication, the 
American Historu^ Review (ahr) selects major articles from other 
journals in the field and cites them in its column of ''Recent Publications. " 
Thus, our first step in developing the list of journals for history was to count 
the number of articles selected from each history journal in a full year ( 1 %9) 

* Once the journals to be analyzed in a given field had been selected, the adjustment critena 
for foreign journals were not applied to those periodicals remaining in that' field. Thus, the 
denominators reflecing the total citations and total source articles have not been completely 
adjusted for the presence of foreign journals. As a result the cumulative percen'^ge of citations 
reported in these tables represents a conservative estimate of the coverage of quality work by 
American authors. ^ 



TABLE Journals Selected in History 



Title 


Selected (1969)« 


Source .Articles (1969) 


1 . Agricultural History 


35 


40 


2. Journal of Economic History 


35 


31 


3. Current History 


30 


88 


4. Church History 


30 


^ 31 


5. American Heritage 


28 


58 


6. Journal of the History of Ideas 


27 


37 


7. Historian 


27 


26 


8. William and Mary Quarterly 


26 


25 


9. Business History Review 


26 


24 


10. American Historical Review 


24 


24 \ 


1 1 . Hispanic American Historical Review 


23 


20 


12. Journal of American History 


21 


27 


13. American Quarterly 


20 


37 


14. Pacific Historical Review 


20 


22 


15. American Archivist 


17 


24 


16. Labor History 


17 


26. 


17. Journal of Church and State 


11 


22 


18. isis 


16 


36 


19. American Neptune 


16 


21 


20. Journal of Modern History 


15 


19 


21. Journal of Southern History 


11 


17 


22. French Historical Studies 


8 


24 


23 . History and Theory 


8 


25 


• Articles published in 1969 that were cited m the 


American Historic at Re 


view\ "Recent Pubhcations'' 



section l969-i'1571 
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of AHR. This count yielded2 12 journals from which one or more articles had 
been cited by ahr. 

Time constraints and the nature of the, study precluded using every 
journal in the ahr bibliography, so the following types were eliminated: 
foreign-language publications, organizational history journals (e.g., 
Quaker History), state history journals (e.g., Georgia Historical Quarter- 
ly), local history journals (e.g., Rochester History), history journals related 
to certain countries (e.g., Liberian Studies Jowr/ia/), journals whose arti- 
cles may be of interest to historians but that deal primarily with other topics 
(e.g.. Southern Speech Journal), and certain types of specialized history 
journals. 

The total list of journals (except for the excluded journals just men- 
tioned, and without either the associated counts or an indication of how 
exactly they had been selected) were sent to three historians, who were 
asked to rate each journal in terms of whether it had major impact, minor 
impact, or no impact on the field of history. The results of this analysis 
substantiated the rank-ordering reflected in the AHR measure. Con- 
sequently^ with one or two minor adjustments based on the reactions of 
these historians, the leading ^3 journals in history as reflected in the AHR 
measure were selected for analysis. 

Table B-4 presents the list of history journals selected, along with the 
measures developed in the study for rating and evaluating those journals. 



Appendix C 

Extra Analyses 



A series of special analyses were conducted as a way of gaining additional 
perspectives on the Science Development program. They are described 
below. 

ZERO-YEAR ANALYSIS 

The argument can be made that plotting data on a year-by-year basis 
obscures effects because all grants were not awarded during the same year. 
That is, 1968 is the third year of some grants, the second year of othWs, and 
the first year of still others. To test this idea we conducted a special analysis 
in which the publication data for each of two fields, physics and chemistry, 
were realigned such that the year the grant was awarded consistently was 
defined as the "zero year." For each school data from the year after the 
grant was awarded were plotted at year "+ 1" regardless of which actual 
calendar year it had been; prefunding data from the year before the award 
were plotted as year' — 1 etc. Then the data were aggregated for the USD 
and the medium control groups. We hoped to find previously obscured 
trends that reflecteda consistent lime lag between funding and subsequent 
publication. The experiment failed. We found the opposite: Effects that 
previously had been revealed were blurred and obscured in this analysis. 
Thfc main reason was that the overall base rate in the field, as reported in 
Table 6-1 , is a powerful factor in these curves. Thus, in doing the zero-year 
analysis we were placing the peak year of 1970 at different points, some- 
times at year "+2, sometimes at year '*4-3,'' etc. 
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FEDERAL SUPPORT TO DEPARTMENTS 

Some data on federal support to individual departments were available 
from the National Science Foundation for the fields of mathematics, 
physic's, and chemistry. In addition to use of the 1968 information as an 
indicator of overall department funding in the residual analyses, a limited 
study was made of trends in this measure. The study had to be hmited 
because the information was available for only four years: 1968, 1970, 1971, 
and 1972. The data were plotted in the hope that this mi^t shed some light 
on the degree to which the momentum begun under Science Development 
funding might have resulted in increased federal support for the recipient 
departments. In all three fields, the USDs were receiving more funds than 
the medium controls in 1968. ahis is not surprising since this measure 
included Science Development funds.) In chemistry the gap remained 
roughly the same in both the public and private sectors through 1972; m 
mathematics and physics the gap widened, an indication that the recipient 
departments in those fields were successful in attracting even more federal 
funds. Further inspection revealed that this trend was concentrated strictly 
in the public sector. In the private sector the gap narrowed by 1972. 



CHANGES IN THE ACE FACULTY QUALITY RATINGS 

A special comparison was made of the Cartter and Roose-Andersen scores 
of recipient institutions since the former was measured in a prefunding year 
and some have proposed the second as a crude measure of postfundmg 
status. Of course, the USD and medium control groups began with virtually 
equal Cartter scores, since this was the basis on which the control group^^^ 
was selected. The analysis revealed no significant change. The Roose- , 
Andersen scores of the two groups also were very close in all three fields. It 
appears from the other analyses in this volume that positive changes 
resulting from the grant occurred after a time lag such that the impact on the 
department's reputation simply could not be felt by the time the Roose- 
Andersen survey was taken. 



IMPACT OF FUNDS FOR PERSONNEL, EQUIPMENT, AND FACILITIES 

The amounts received by each recipient department for personnel, equip- 
ment, and facilities were added to the master file. As the reader will recall, 
in each of the multivariate analyses described in the text several partial 
correlations were computed, e.g., a partial between the criterion and a 
dummy variable indicating whether or not the school received a grant. At 
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the same time three other partial correlations were generated to indicate the 
relative impact of the funds for personnel, equipment, and facilities. An 
analysis was conducted of this set of partials. The goal was to trace 
differential effects of funds for these three purposes— patterns that could 
serve as aguide for future federal funding policy. Without boring the reader 
with the gory details, suffice to say that this analysis was a disappointment. 
The pattern of these partial correlations did not yield new insights, although 
It was consistent with common sense expectations. 



SCHOOL-BY SCHOOL ANALYSIS: THE THRESHOLD THEORY 

Once the entire master file of institutional data had been assembled, indi- 
vidual graphs were constructed to map changes in key variables for each 
field at each funded univeftity, e.g., the. chemistry department at the 
University of Maryland, Each graph plotted the data we had assembled on 
that institution over the 15-year period: faculty size, publication rate gener- 
ated by the multivariate analysis, and so forth. The graphs for all USD 
recipients then were rank-ordered, to separate the "winners" from the 
*Mosers," on the basis of the departmental residual scores. The goal was to 
see if common factors could be detected that differentiated these two 
groups. Identification of such common factors might help future funding 
programs anticipate which schools would profit most from institutional 
support. 

The only factor uncovered was the size of the Science Development 
grant, which, in turn, led to the hypothesis of a threshold effect; that is, it 
appeared that departments receiving above a given amount in the initial 
Science Development grant were much nr.ore likely to perform well, while 
thote below that amount tended to do poorly. The amount varied from 
fiejd to field (an iniUal award of $700,000 for mathematics and $800,000 
foiiphysics and<rhemistry.) As a follow-up on this notion, the USD 
reclpientsirreach field were dichotomized at the threshold, and the trends 
on the criteria were analyzed separately for the two new groups. In Figures 
C-1 , C-2, and C-3, publication data for those two groups, as well as for the 
medium control groups are plotted. Note that the **above threshold" 
USD recipients perform brilliantly. Tor example, in mathematics these 
schools seem to be moving into a catagory of excellence; they catch the 
high controls, a finding that is rarely matched elsewhere in this volume. In 
chemistry, the USD schools show a definite effect on publication rates, 
where before there was virtually none. Of the two experimental fields, 
chemistry had a largei^ortion of recipients with below threshold funding, 
suggesting one possible explanation for the poorer performance in this field 
reported eariier. 
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USD Institutions 
(Over $800,000) 




1958 1960 1962 1964 1966 1968 1970 1972 

YEAR 



FIGURE C*3 Thrtshotd analysis of ckpartmtntal publication rates in science development 

and control institutions: chemistry. 



PUBUCATION PER PERSON VS. FACULTY SIZE 

Finally, we conducted an additional analysis in which publications per 
faculty member was plotted vs. faculty size for the four time points on 
which the size data were available. Examination of the slopes from these 
curves, while done with caution, shed some light on the mechanisms 
involved in the changes in productivity. The reader easily can replicate 
these graphs from the data presented in Chapters 5 and 6. 

Extremely steep slopes indicate that per person productivity rose shar- 
ply while departmental size remained about the same; on the other hand, a 
relatively flat line indicates that per person productivity increased only 
sligfttly as the department grew. Inspection of these curves is a way of 
determining whetlier increased per person productivity is simply a mono- 
tonic function of larger departmental sizes or the result of having ob- 
tained a "critical mass." 

Our conclusion based on these graphs and the review of the residual data 
discussed above is simply that the situation differed radically from school 
to school. A variety of patterns were found, including some funded (and 
control) departments in which per person productivity rose sharply while 
size increased slightly, indicating excellent departmental leadership. 
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